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Abstract 

Thermal contact in permanent regime is the archetype of non-equihbrium stationary states 
driven by constant non-equihbrium constraints enforced by reservoirs of exchanged conserved 
microscopic quantities. In the present paper we are interested in a class of models with a 
finite number of possible configurations where the heat exchanges are described as changes in 
the populations of energy levels when the configurations stochastically evolve under a master 
equation. Systems with discrete configurations have no microscopic Hamiltonian dynamics 
but, if the microscopic deterministic dynamics which conserves energy is assumed to be er- 
godic, it is shown that the transition rates obey a modified detailed balance (MDB) which 
is generically expressed in terms of the microscopic exchange entropy variation, namely the 
opposite of the variation of the thermodynamic entropy of the reservoirs under a microscopic 
jump of the system configuration. We investigate the generic statistical properties for measur- 
able quantities which arise from the MDB constraint. For a finite-time evolution (transient 
regime) from an initial equilibrium state we derive a detailed fluctuation relation for the 
excess exchange entropy variation and an associated integral fluctuation relation. In the non- 
equilibrium stationary state (long-time regime) , the proper mathematical definition of a large 
deviation function is introduced together with alternative definitions, and fiuctuation relations 
are derived. The fluctuation relation for the exchange entropy variation is merely a particular 
case of the Lebowitz-Spohn fluctuation relation for the action functional [T]. The infinite time 
limit of any odd cumulant per unit time of an exchanged quantity is expressed in terms of a 
series involving higher even cumulants and powers of the thermodynamic force associated to 
the mean current ; every relation can be seen as a generalized Green-Kubo relation valid far 
from equilibrium. Its generalization to the case of several independent macroscopic currents 
is derived. It entails a relation between the nth-order non-linear response coefficient of any 
odd cumulant per unit time in the vicinity of equilibrium and a finite number of lower-order 
non-linear response coefficients of even cumulants per unit time. The latter relations can be 
seen as another kind of generalizations of the standard Green-Kubo formula pertaining to the 
first-order coefficient. 
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1 Introduction 

1.1 Issues at stake 

Thermal contact between two energy reservoirs is one of the first issues addressed by early ther- 
modynamics and its second principle about entropy variation. It has been revisited in the context 
of the description of transport phenomena either from the statistical approach by the Boltzmann 
equation and its BBGKY hierarchy generalization or from the plienomenological thermodynamics 
of irreversible processes [2J [31 H] and the local equilibrium plienomenological approach for inho- 
mogeneous continuous media [S]. Until recently there have been repeated attempts to formulate 
thermodynamics for non-equilibrium steady states [SI [7] . Nowadays the statistical description for 
the fluctuations of heat exchanges between a small system and two thermal baths is part of the 
so-called " stochastic thermodynamics" of small systems incorporating the effects of fluctuations 
[8l[9]. In the last two decades the latter domain has been the subject of an increasingly intense 
activity, from the theoretical as well as from the experimental point of view with very fast tech- 
nological improvements ]W[ [TT] . (For a pedagogical introduction in a nutshell see for instance 
Ref.|12|. A recent extended review is to be found in Ref.|9J). 
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First steps in out-of-equilibrium statistical mechanics have been the study of the hnear (static 
or dynamic) response to an external constraint that drives the system out of its equilibrium state. 
This vast topic began with the Einstein fluctuation-dissipation relation for Brownian motion (see 
collected translated papers in [T^) and is still active (for a review see [MJ). Nowadays, the question 
of linear response in the vicinity of a non-equilibrium steady states is under investigation (see for 
instance [fSJ [TBJ [T71 [TS|), but the subject is beyond the scope of the present work. 

Non-perturbative approaches for systems far from equilibrium have followed various ways. One 
trend has been the search for a unifying variational principle based on a large deviation point of 
view, which would generalize the entropy-probability relation introduced by Einstein in his theory 
of thermodynamic equilibrium fluctuations |19) in order to explain the opalescence phenomenon. 
Indeed, principles of equilibrium statistical mechanics can be seen as resulting from the maxi- 
mization of the Shannon entropy under various constraints about macroscopic observables in the 
framework of information theory |20l \Tr\ . From this point of view, statistical mechanics, which 
allows one to retrieve the laws of thermodynamics while describing mesoscopic fluctuations, may 
be interpreted as an example of a large deviation theory, which provides the probabilities for an 
observable to deviate from its most probable value [12] . In order to go beyond the phenomenolog- 
ical irreversible thermodynamics, Oono [23] has promoted the idea of applying the large deviation 
theory to the non-equilibrium statistical physics seen as a statistical mechanics along the time 
axis. In this direction efforts have been devoted to the study of the maximization of the Shannon 
entropy for the system histories under some constraints, such as fixed values for the macroscopic 
out-of-equilibrium currents in a a non-equilibrium steady state (see |241l25j and references therein). 

In the same spirit a second path, the quest for some relevant physical quantities that would 
obey some universal principle, is the subject of the fluctuation relations either at finite time 
(transient regimes) or in the infinite-time limit (stationary regimes) . The active topic of fiuctuation 
relations was initiated by works about a symmetry of the large deviation function of some entropy 
production rate in the non-equilibrium stationary states of some chaotic dynamical systems [2611271 
[28, 2i)J , and about the Jarzynski identity for the finite-time cumulated work when a Hamiltonian 
system is driven out of an initial equilibrium state by the variation of some external parameter 
[30| . The first results pertained to classical systems but fluctuation relations in quantum systems 
have also been investigated (see for instance the reviews |.31, 32J). A short list for successive steps 
on the narrower pathway of fluctuation relations for systems with Markovian stochastic dynamics 
is given by the following references [331 IMl [551 [551 [2 [371 ISHl IHl ISl- Among these results, in the 
more speciflc case of systems with stochastic evolutions described by a master equation for the 
probability of the system conflgurations, the milestone is the fluctuation relation obeyed by the 
dimensionless action functional introduced by Lebowitz and Spohn |IJ. 

In the even more restricted case where the system exchanges conserved microscopic quantities, 
such as energy quanta or particles or elementary volumes, with infinite size reservoirs which stay 
in their equilibrium states during the considered evolution of the finite system, the transition 
rates must obey a relation, which will be referred to as the Modified Detailed Balance (MDB)Q. 
When the transition rates obey the MDB, the action functional introduced by Lebowitz and Spohn 
coincides with the variation of the sum of the reservoirs thermodynamic entropies. Then, apart 
from the fluctuation relation obeyed by the latter entropy variation, the known results about large 
deviation functions which are the most relevant for the topic of the present paper deal with the 
following quantities in flnite size systems : the heat current which goes through the flnite system 
which sets up a thermal contact between two thermostats [40l |4j (also flrst addressed in the 
context of classical or quantum Hamiltonian dynamics in Ref . [42| ) : more generally the various 
macroscopic currents through a system in a non-equilibrium stationary state sustained by various 
kinds of reservoirs |43t ,44] . 

Apart from general approaches, a third trend in the quest for some non-perturbative statistical 
theory of out-of-equilibrium phenomena has been the search for solvable models which could give 
some hints in the comprehension of these phenomena in the absence of any theoretical framework. 
We are interested in a class of models where the system has a flnite number of possible conflgura- 

^The denomination "generalized" detailed balance is also to be found in the literature. 
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tions, the heat exchanges are described as changes in the populations of its energy levels, and the 
configurations stochastically evolve under a master equation with transition rates bound to obey 
the MDB relation arising from the existence of an underlying ergodic deterministic microscopic 
dynamics which conserves energy. In paper II |45) we perform explicit analytical calculations for 
the very simple case where the thermal contact system is reduced to two spins, each of which is 
flipped by a single thermostat, the two thermostats being at different temperatures. 

1.2 Main results valid beyond the thermal contact example 

In the present section we point out the main ideas and generic results of our study when the system 
is driven out of equilibrium by exchanges of some conserved microscopic quantities between the 
system and reservoirs characterized by constant thermodynamic intensive parameters. For the 
sake of pedagogy, we express them in the language of the thermal contact example. The general 
arguments and results in the situation where there are more than two reservoirs and/or several 
kinds of exchanged quantities are more detailed in section |6l 

In the whole paper we put emphasis on the exchange entropy variation Acxch'S' of the system, 
which is opposite to the well-defined entropy variation of the reservoirs described in the thermo- 
dynamic limit, because it seems to be the crucial quantity to consider in order to answer the 
key-point question of identifying the relevant measurable quantities which are to obey non-trivial 
universal properties (beyond long-time decorrelations and the subsequent existence of large devia- 
tion functions) . This point of view slightly differs from the common picture where the focus is put 
on the variation of the Shannon-Gibbs entropy of the system. The latter picture has arisen from 
the seminal works by Crooks |351 1361 146j and has led to the notion of entropy production along a 
stochastic trajectory of the system microscopic configurations (see [3^ ^ for a recent formulation) . 
In the whole paper Boltzmann constant is set to 1 and S denotes dimensionless entropies. 

We point out that Appendix |E] contains the proper mathematical definition of the large de- 
viation function, and some variants for the models at stake, together with demonstrations of 
properties about large deviation functions which are not usually exhibited in the literature of the 
physicists community. 

1.2.1 Constraints from ergodicity and energy conservation 

We consider models for thermal contact where the energy exchanges between several very large 
systems with indices a's are realized through interactions with a system which has a finite number 
of configurations, namely which has discrete variables in finite number. The latter system is 
referred to as the "spin system" in the following. The spin system has an internal energy which 
is bounded, since its configuration space is finite. When each large part with index a is described 
only at a macroscopic level, i.e. only by its energy Ea , we have to resort to a statistical description, 
where the knowledge of the system is specified by the following datas : the energies of the large 
parts and the configuration C of the spin system. Such a set of datas defines what we call a 
mesoscopic configuration of the total system in the sequel. Then the stochastic evolution is 
determined by transition rates between these mesoscopic configurations. The key points are the 
following. 

First we have to answer the question : which constraints must be obeyed by the choice for the 
transition rates ? Discrete variables cannot evolve under a microscopic Hamiltonian dynamics, but 
the existence of equilibrium in the infinite-time limit is ensured if the deterministic microscopic 
dynamics is ergodic and conserves the sum of the energies of every part. In a given energy level 
all ergodic microscopic dynamics have the same period and the interaction pattern entails that, 
during a period of the microscopic dynamics, the number of transitions which occur from one 
mesoscopic configuration of the total system to another one is equal to the number of transitions 
in the opposite sense (subsection 12.11 and Appendix |A|. As a consequence, if the mesoscopic 
dynamics is approximated by a Markov process according to the prescription derived in Appendix 
IbI then the corresponding transition rates must satisfy three constraints (subsection 12. 2p . In 
particular the transition rates at the mesoscopic level obey the microcanonical detailed balance 
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(j2.9p . (For a comparison, the derivation of the microcanonical detailed balance in the case of 
continuous mesoscopic variables and an underlying microscopic Hamiltonian dynamics invariant 
under time reversal is presented in Appendix [C]) . The interaction pattern further transforms the 
microcanonical balance into the reduced expression (j2.1ip which involves the Boltzmann entropy 
variation of the large part which exchanges energy in the transition. 

In the limit where the sizes of the large parts go to infinity before the time evolution of the 
system is considered, the global system does not reach equilibrium but the large parts are in the 
thermodynamic equilibrium state which they would tend to in the infinite-time limit if they were 
isolated (subsection 12. 3p . In the case of thermal contact the large parts behave as thermostats 
characterized by their inverse temperatures /3a 's. Then the previous constraints for the transition 
rates between mesoscopic configurations become three conditions which must be obeyed by the 
transition rates between the spin system configurations in the corresponding transient regime. 
Let (C'|W|C) denote the transition rate from configuration C to configuration C in the transient 
regime. The three conditions are the following ones. First, the set of transition rates must be such 
that any configuration C can be reached by a succession of jumps with non-zero transition rates 
from any configuration C, namely in the network representation of the stochastic evolution 

the graph G associated with the transition rates is connected. (1-1) 

In other words the Markov matrix M defined from the transition rates by (|2.15p must be irreducible 
and the property will be referred to as the irreducibility condition. Second, the transition 
rates must obey the microscopic reversibility condition for any couple of configurations (C,C'), 
namely 

(C'|W|C)7^0 ^ (C|W|C')7^0. (1.2) 

Third, the transition rates have to satisfy the modified detailed balance (MDB) which takes the 
form (|2.16p in the case of thermal contact. 

We choose to reexpress the MDB relation between transition rates in terms of the exchange 
entropy variation (5exch'S'(C C) associated with a jump of the spin system from a microscopic 
configuration C to another one C: 5exchS{C' C) is the opposite of the microscopic variation of 
the sum of the thermodynamic entropies of the heat baths when a single thermostatchange the 
spin system energy by flipping a spin. The MDB relation (|2.16p for the thermal contact reads 
quite generally 

(CMC) s....sic'^o .,3. 

[CMC] ■ ^ ^ 

This formulation of the MDB is the most generic statement, because it also applies to exchanges of 
other kinds of conserved microscopic quantities, such as particle numbers or elementary volumes, 
and it may involve simultaneous exchanges of different kinds of conserved quantities. We stress 
that ScxchS{C' -i— C) is defined unambiguously and involves experimentally measurable quantities 
(in the thermal contact case the inverse temperatures of the energy reservoirs /3a 's as well as the 
heat amounts received by the system). 



1.2.2 Various entropy variations inherent to Markovian dynamics 

In section [3] we stress some consequences of the fact that the configuration probability obeys a 
master equation. First, even if the transition rates do not obey the MDB, the irreducibility of the 
Markov matrix implies the uniqueness of the stationary state and the role played by the relative 
entropy with respect to the stationary solution, S'i.oi[-P(i)|-Pst] is recalled (subsection 13. ip . We 
introduce the generic currents (|3.10l) - p.lll) for a microscopic variation when the system goes out 
of a given configuration C ; the averages of such currents show off either in the mean energy 
time derivative or in the mean exchange entropy flow (subsection 13. 2p . In order to make the 
comparison with the thermodynamics of irreversible processes, the time-derivative of the Shannon- 
Gibbs entropy S^'^[P{t)] of the system is split into two contributions, an exchange (or external) 
part dexchS/dt arising from exchanges with the external reservoirs, hereafter called the exchange 
entropy flow, and an internal (or irreversible) part dintS'^'^/dt due to the internal irreversible 
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processes in the system, also called the entropy production rate fsubsection l3.3p . In the stationary 
state, the time derivative dS^'^[Pst]/dt vanishes and the entropy production rate becomes opposite 
to the exchange entropy flow. Henceforth, in the case where there are only two reservoirs of 
energy, the stationary entropy production rate can be rewritten as in phenomenological irreversible 
thermodynamics, namely 



dcKchS 



dt 



dintS^ 



dt 



= TJ (1.4) 



where J is the mean heat current which goes through the system from the heat bath 2 to the heat 
bath 1 and = /3i — /32 is the associated "thermodynamic force". When the MDB is satisfied 
the expressions for dcxchS/dt and dintS^° / dt coincide with the splitting of dS^° /dt introduced in 
Ref.[T]. Moreover, the entropy production rate appears as the sum of two positive contributions : 
on the one hand, the opposite of the time-derivative of the relative entropy S'rei[f'(i)|^'st] and, on 
the other hand, the average of a microscopic current associated to some stationary affinity variation 
when the system jumps out of a configuration. The latter current proves to be positive even before 
averaging over the configurations (see (j3. 401) 1. For the sake of completeness we recall how, in the 
graph theory where the master equation is represented by a network, a non equilibrium stationary 
state is characterized by the affinities and probability currents associated with a restricted number 
of cycles defined from the network. When the graph is a pure cycle, there is a single affinity which 
has a simple probabilistic interpretation given in paper II |45| . 



1.2.3 MDB and non-perturbative symmetries in transient regimes 

In the form p.3p where it involves the microscopic exchange entropy variation (5oxchS'(C' C) 
associated with a jump from configuration C to configuration C, the MDB entails time-reversal 
symmetries for finite time intervals at more and more mesoscopic levels as follows. The ratio (|4.6p 
of the probabilities for a microscopic history "Hist and the time-reversed one is determined by the 
cumulated exchange entropy variation Aoxch'5'['Hisi] along a history defined in (|4.1I) . Then, the 
ratio (|4.8p between, on the one hand, the probability for all evolutions with fixed initial and final 
configurations and given heat amounts Qi and Q2 received from the thermostats and, on the other 
hand, the probability for all backward evolutions with exchanged initial and final configurations 
and opposite heat amounts — Qi and — Q2 is determined by the exchange entropy variation 

Aexch5(Ql,Q2)=/3lQl+/?2Q2. (1.5) 

When the system is prepared in an equilibrium state at the inverse temperature /3o and suddenly 
put in thermal contact at the initial time of measurements with two heat baths at the inverse 
temperatures /3i and /32, the symmetry (j4.8p together with the specific form of the equilibrium 
canonical distribution lead one to consider the following measurable quantity : the excess exchange 
entropy variation Ag^lj^'^" (Qi , Q2) defined as the difference between the exchange entropy variation 
under the non-equilibrium external constraint /3i 7^ /32 and that under the equilibrium condition 
/?! = /32 = /?o, namely 

A:;;:^^ "(Qi, Q2) = Aexch5(Qi, Q2) - MQi + Q2). (1.6) 
^exch'^''(2i' 22) obeys the symmetry relation at any finite time, or "detailed fluctuation relation", 

^-"V ^) ^e-^'"', (1.7) 

The latter relation itself entails the identity, or "integral fluctuation relation", {e^i'^^h^" ^) ^iig — 1 
in the spirit of Jarzynski identity |30j . To our knowledge these two relations have not appeared 
explicitly in the literature, though they may be seen as a transposed application of the argument 
first exhibited by Crooks [3B] and then Seifert [SSJ |H] for the entropy production along a stochastic 
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trajectory when the system is in thermal contact with only one heat bath and is driven out of 
equilibrium by a time-dependent external parameter. We notice that if the configuration prob- 
ability distribution in a stationary state of the system happens to be the canonical distribution 
at an effective inverse temperature /32), as it is the case in the solvable model considered 

in paper II f451, there exist similar detailed and integral fluctuation relations for a protocol where 
the system is initially prepared in a non-equilibrium stationary state with two heat baths at the 
inverse temperatures /3i and and suddenly put at the initial time of measurements in thermal 
contact with two heat baths at the inverse temperatures /3i and /32- Then ^°^yI^°{Qi, Q2) is to 
be replaced by A^^^^''^* (Qi, Q2) where /3° is the effective inverse temperature /32)• 
1.2.4 MDB and fluctuation relations in the stationary regime 

For a system with a finite number of configurations, when the Markov stochastic matrix for the 
continous-time evolution of the configuration probabilities is irreducible, (see definition (jl.ip ). 
the Perron-Frobenius theorem can be applied: the system has a single stationary state, and it is 
such that every configuration has a non- vanishing weight (see for instance Ref. HZ]). Moreover 
the system reaches its stationary state in a exponentially-short time f48]. Then the symmetry 
relation (j4.8p enforced by the MDB at finite time leads to the existence of lower and upper bounds 
for the ratio between the finite-time probability to measure heat amounts Qi and Q2 and the 
corresponding probability for the opposite values, when the system is in its stationary state (see 
(IS.ip V Similar bounds are exhibited in Ref.j41j. 

The Markovian property of the evolution implies that the cumulated heats Qi and Q2, and 
subsequently the cumulated exchange entropy variation Acxch'5', all grow linearly with time in 
the long-time limit and that there exist large deviation functions for each of them |49[ [50] . The 
proper mathematical definition of a large deviation function is recalled in Appendix lE.il and other 
alternative definitions when exchanged quantities are discrete are introduced in Appendix IE. 21 

The finite-time inequalities (|5.ip entail a symmetry in the long-time limit, according to the 
general results derived in Appendix IE. 31 : the dimensionless exchange entropy variation Aoxch'5' 
obeys the fiuctuation relation 

/a_,s(J) - /a_„s(- J) = -J, (1.8) 

where denotes the values taken by the cumulated current Aoxch*S'(i)/t. The fluctuation relation 
for Acxch'S' is a special case of the more general fluctuation relation for the action functional 
introduced by Lebowitz and Spolm [IJ ; indeed when the transition rate obey the MDB, the action 
functional for a given history becomes equal to the entropy variation of the heat reservoirs, namely 
to the opposite of the exchange entropy variation. 

For a system with a finite number of configurations, Qi + Q2 is bounded. Then, according to 
the results of Appendix IE. 41 the large deviation function for Q2 is equal to that for — Qi , while 
the fiuctuation relation (|1.8p for fA^^^^S also entails a fiuctuation relation for the large deviation 
function /g^ for the cumulated heat Q2, because the difference between AexchS' and — (/3i — /32)Q2 
is finite at any time t. The fluctuation relation can be written in the generic form 

fQAJ)-fQ2{-J)^J'J, (1-9) 

where J- is the thermodynamic force conjugated to the mean current J — limt_j.-|_oo (i/(t)) = 
limt_5.+oo (Q2(^))/i through the expression of the exchange entropy flow in the stationary state 
(see (HH)). 

1.2.5 MDB and generalized Green-Kubo relations 

Quite generally, when there exists a large deviation function for the cumulated current J7t = Xt jt 
associated with the cumulated quantity X(, the generic expression of the linear response in the 
non-equilibrium state far from equilibrium is given in (|5.20p : it relates on the one hand, the 
coefficient dJ j dT of the linear response of the heat current J to a variation of the thermodynamic 
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force T and, on the other hand, the infinite-time hmit of the variance per unit time for the 
cumulated quantity Xt in the non-equihbrium steady state, with a coefficient whose expression 
depends on the system. 

In the hmit where the thermodynamic force J- vanishes, dJ jdJ- tends to the hnear-response 
coefficient near equihbrium, namely the kinetic Onsager coefficient, L = dJ / dJ-\jr^a- If the system 
obeys the fluctuation relation (|1.9p . with Xt in place of Q2, then the coefficient in the identity 
(|5.20p takes the universal value ^ and the identity becomes the fluctuation-dissipation relation 
(also referred to as the Green-Kubo relation) between the kinetic Onsager coefficient and the 
infinite-time limit of the variance per unit time of Xt at equilibrium. In the case of thermal 
contact, in the limit where the difference /3i — /32 between the inverse temperatures vanishes, the 
ratio between the stationary heat current which goes through the system from the heat bath 2 to 
the heat bath 1 and the difference /3i — (32 becomes equal to i times the variance per unit time of 
the heat amount exchanged with one thermal bath at equilibrium. (When the total system is at 
equilibrium, the net heat amount Qi -\~ Q2 received by the system remains finite at any time, but 
the heat amount received from one bath grows linearly in time in the long-time limit). 

For a system with a finite number of configurations, the MDB entails a symmetry upon the 
generating function for the infinite-time limit of the cumulants oi Xt per unit time. This symmetry 
takes the generic form (j5.54p in terms of the thermodynamic force J'. We then show that, in the 
infinite-time limit, any odd cumulant per unit time can be expressed in terms even-order 

cumulants per unit time through the relation 

lini ^ kii^y4^2fc+i ^ Zl forn = 0,l,... , (1.10) 

fe=0 

where c?fe is given in (I5.64p . The corresponding expressions for the ratios (1/ J") x hmt^+00 Kl^n+iJ jt 
may be viewed as generalized Green-Kubo relations valid both far from equilibrium and for all 
cumulants . 

We also express the response of any odd cumulant per unit time at any order in the thermody- 
namic force T near equihbrium in terms of non-linear response coefficients for even cumulants per 
unit time at lower orders in T near equilibrium (see (j5.70p l. At first order one gets the generalized 
fluctuation-dissipation relations (j5.7ip for any odd cumulant. 

In the more general situation where there are several independent mean currents between 
reservoirs, we derive the corresponding generalized Green-Kubo relations (|6.2ip . From the latter 
equations one can derive relations between non-linear response coefficients. These relations have 
already been settled by another method by Andrieux and Gaspard [FT]. As noticed by these 
authors some of them lead to symmetries which are generalizations of the Onsager reciprocity 
relations. 



2 Constraints upon transitions rates 

In the present section we review some of the constraints that ergodic deterministic energy-conserving 
microscopic dynamics puts on the statistical mesoscopic description of a finite system which es- 
tablishes thermal contact between energy reservoirs. 

Indeed, the following situation occurs commonly : the interactions in the whole system allow 
to define one small part in contact with otherwise independent large parts. The large parts, which 
involve a huge number of degrees of freedom, do not interact directly among each other (this 
gives a criterion to identify the distinct large parts), but are in contact with the small part, which 
involves only a few degrees of freedom. Moreover each degree of freedom in the small part is 
directly in contact with at most one large part. This results in a star-shaped interaction pattern. 
It is convenient then to forget about the microscopic description of the large parts, and turn to 
a statistical description of their interactions with the small part. Some general features of the 
statistical description can be inferred from microscopic ergodicity. 



9 



2.1 Ergodicity 



2.1.1 Ergodicity in classical Hamiltonian dynamics 

In classical mechanics, the time evolution of a system in phase space is described by a Hamiltonian 
H. If the system is made of several interacting parts, the Hamiltonian splits as H — H^cc + ^^intj 
where i/dcc accounts for the dynamics if the different parts were decoupled and Hi^t accounts for 
interactions. The energy hypersurface H — E, usually a compact set, is invariant under the time 
evolution, and in a generic situation, this will be the only conserved quantity. 

The ergodic hypothesis states that a generic trajectory of the system will asymptotically cover 
the energy hypersurface uniformly. To be more precise, phase space is endowed with the Liouville 
measure (i.e. in most standard cases the Lebesgue measure for the product of couples made by 
every coordinate and its conjugate momentum), which induces a natural measure on the energy 
hypersurface, and ergodicity means that in the long run, the time spent by the system in each 
open set of the energy hypersurface will be proportional to its Liouville measure. Ergodicity can 
sometimes be built in the dynamics, or proved, but this usually requires immense efforts. 

Ergodicity depends crucially on the fact that the different parts are coupled : if _ffi„t = 0, each 
part will have its energy conserved, and motion will take place on a lower-dimensional surface. If 
Hint is very small, the system will spend a long time very nearby this lower-dimensional surface, 
but at even longer time scales, ergodicity can be restored. By taking limits in a suitable order (first 
infinite time and then vanishing coupling among the parts) one can argue that the consequences 
of ergodicity can be exploited by reasoning only on H^cc- 

Notice that in this case, we have in fact some kind of dichotomy : H^ec defines the energy 
hypersurface, but cannot be used to define the ergodic motion, which is obtained from H^cc + ^^int 
via a limiting procedure. So the dynamics conserves -ffdec but is not determined by i?dec- 

2.1.2 Ergodicity in deterministic dynamics for discrete variables 

Our aim is to translate the above considerations in the context of a large but finite system described 
by discrete variables such as classical Ising spins. 

In the case of discrete dynamical variables, one can still talk about the energy E oi a configu- 
ration, but there is no phase space and no Hamiltonian dynamics available. So there is no obvious 
canonical time evolution. This is where we exploit the previously mentioned dichotomy: we do not 
define the time evolution in terms of E, but simply impose that the deterministic time evolution 
preserves E^cc and that it respects the star-shaped interaction pattern between the small part and 
the large parts. 

Ergodicity is most simply expressed if one assumes that time is discrete as well. Then deter- 
ministic dynamics is given by a bijective map, denoted by T in what follows, on configuration 
space, applied at each time step to get a new configuration from the previous one. As the config- 
uration space is finite, the trajectories are bound to be closed. Then, for a given initial value E of 
i?dcc 7 a specific dynamics T conserving i?dcc corresponds to a periodic evolution of the microscopic 
configuration of the full system inside the energy level -Bdoc = E. 

Ergodicity entails that the corresponding closed trajectory covers fully the energy level E^cc — 
E, and then it must cover it exactly once during a period because the dynamics is one to one. As 
a consequence the period of the ergodic evolution inside a given energy level of -Edoc is the same 
for all choices of ergodic dynamics T which conserves i?dcc- This period, denoted by N in time 
step units, depends only on the value E of i?dec, 

N = Qd,,{E), (2.1) 

where Udec{E) denotes the total number of microscopic configurations in the level i?dec = E. This 
is reminiscent of the microcanonical ensemble. Let us note that in classical mechanics, there is a 
time reversal symmetry, related to an involution of phase space, changing the momenta to their 
opposites while leaving the positions fixed (see (jC.2[) '). In the discrete setting, involutions J such 
that JTJ = always exist, but there is no obvious candidate among them to represent time 
reversal and draw conclusions from it. 
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In the context of discrete variables the star-shaped interaction pattern is implemented as 
follows. We may naively assume that the energy conserved by the dynamics T is simply i^dec, 
as if there were no energy for the interactions between the small part and the large ones, but T 
must reflect the fact that the large parts interact only indirectly: there is an internal interaction 
energy £{C) for every configuration C of the small part and each change in the small part can be 
associated to an elementary energy exchange with one of the large parts. If the small part can 
jump from configuration C to configuration C in a single time step by exchanging energy with the 
large part a, we use the notation C' = FqC In this configuration jump the energy i?doc of the 
global system is conserved and the energy of the large part a is changed from Ea to E'^ according 
to the conservation law 

f-[^(C')-^(C)] if C'^F.C ^^^^^ 
I otherwise, 

while the energies of the other large parts are unchanged. Apart from these energy exchange 
constraints and from ergodicity, the deterministic dynamics T is supposed to obey some other 
natural physical constraints which will be specified later. 



2.1.3 Constraints from ergodicity and interaction pattern upon coarse-grained evo- 
lution 

We start from the the familiar observation that keeping track of what happens in detail in the large 
parts is out of our abilities, and often not very interesting anyway. Our ultimate interest is in the 
evolution of the configuration C of the small part in an appropriate limit. As an intermediate step, 
we keep track also of the energies Ea& in the large parts, but not of the detailed configurations in 
the large parts. 

The coarse graining, which keeps track only of the time evolution of the configuration C of 
the small part and the energies EaS of the large parts, is defined as follows. To each microscopic 
configuration ^ of the full system we can associate the corresponding configuration C = C(^) of the 
small part and the corresponding energy Ea = Ea{^) carried by part a. To simplify the notation, 
we let E_ denote the collection of EaS, so £^ is a vector with as many coordinates as there are large 
parts. 

As shown in previous subsubsection, if the initial value of the energy -Edec is equal to E, then, 
over a period equal to = f2doc(-E') in time step units, the trajectory of ^ under any microscopic 
ergodic dynamics T corresponds to a cyclic permutation of all the microscopic configurations in 
the energy level £^dcc = E. Thus ergodicity entails that at the coarse grained level, if Ni^q^e) 
denotes the occurrence number of {C,E_) during the period of N time steps, N(^c,e) is nothing but 
^dcc{C,E_), the number of microscopic configurations of the full system when the small part is in 
configuration C and the large parts have energies E^s in the energy level E = E^cc, namely 

N{C,E) = fldcc{C,E) (2.3) 

with 

E ^ Edec = Y,Ea- (2.4) 

a 

In the following we fix the value of E and N is called the period of the dynamics, while the energy 
constraint E ~ £{C) + J2a^a.- is often implicit in the notations. 

Another crucial point is that, since any specific dynamics T under consideration respects both 
the conservation of i^^dcc and the interaction pattern specified at the end of subsubsection !^. 1.21 the 
number of jumps from {C,E) to (C^ E') over the period of N time steps, denoted by A^(c,_e),(c',£;')' 
is equal to the number of the reversed jumps from (C . E') to {C,E) during the same time interval, 
namely 

^{c,e),{C',e;) = ^(C',e!),(c,e)- (2-5) 
In Appendix |X] we give graph-theoretic conditions, not related to ergodicity, that ensure this 
property, and show that they are fulfilled in one relevant example, as a consequence of the star- 
shaped interaction pattern. 
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2.2 Markovian approximation for the mesoscopic dynamics 

2.2.1 Definition of a Markovian approximation for the mesoscopic dynamics 

As already noticed, over the period of iV = Q^cciE) time steps a trajectory under any microscopic 
ergodic dynamics T in the energy level -Edec = E corresponds to a cyclic permutation of the 
N microscopic configurations ^'s in the energy level. Therefore, if the configuration at some 
initial time is denoted by ^i, then the trajectory is represented by the sequence w = ^1^2 ■ ■ ■ 
where ^i+i — T^i with ^n+i — Ci- By the coarse-grained procedure which retains only the 
mesoscopic variable x = {C,E_), the succession of distinct microscopic configurations w is replaced 
by w — X1X2 ■ ■ -xm where Xi = x{£^i) — {C {^i) , E_(^i)) . In w various x^'s take the same value, and 
a so-called transition corresponds to the case x^+i 7^ Xi, namely the case where the configuration 
C of the small system is changed in the jump of the microscopic configuration of the full system 
from to ^i+i = T^i. 

Since the large parts involve many degrees of freedom, the number of times some given value 
^ = {C,E_) appears in the coarse-grained sequence w is huge, and even if uj is given by the 
deterministic rule ^i+i — T^i, there is no such rule to describe the sequence w. Moreover the 
microscopic configuration at the initial time, ^1, is not known so that the coarse grained sequence 
which actually appears in the course of time is in in fact a sequence deduced from w by a translation 
of all indices. 

As explained in Appendix |Bl one may associate to the sequence w a (discrete time) Markov 
chain such that the mean occurrence frequencies of the patterns x and xx' in a stationary stochastic 
sample are equal to the corresponding values, N^/N and N^x'/N, in the sequence w determined 
by the dynamics T (up to a translation of all indices corresponding to a different value of the initial 
microscopic configuration). Whether this Markovian effective description is accurate depends on 
several things: the choice of T, the kind of statistical properties of w one wants to check, etc. 

We may also argue that a continuous time description is enough if we restrict out attention to 
microscopic dynamical maps T's such that transitions, namely the patterns xx' with x' ^ x, are 
rare and of comparable mean occurrence frequencies over the period of A'^ time steps. In other 
words, most of the steps in the dynamics amount to reshuffle the configurations of the large parts 
without changing their energies, leaving the configuration of the small part untouched. The latter 
physical constraint and the hypothesis of the validity of the Markovian approximation select a 
particular class of dynamics T. 

With these assumptions, we associate to the sequence w of coarse grained variables {C,E_) a 
Markov process whose stationary measure shares some of the statistical properties of w, namely 
the values of the mean occurrence frequencies of length 1 and length 2 patterns. The transition 
rate from {C,E) to iC',E') with {C,E) ^ (C',^') in the approximated Markov process is given 
by (jB.4p . where we just have to make the substitutions = N'(^c,e) and N^x' = N(c.e),(c' ,e'): 
while the corresponding stationary distribution by given by (|B.2p . P^{C,E_) = N(c,e)/N with 

N = Y.iC,E)NiC,E)- 



2.2.2 Microcanonical detailed balance and other properties 

By virtue of the ergodicity property (|2.3p at the coarse-grained level, the transition rate in the 
approximated Markov process reads 

W{C',e: ^C,E) = ^n~^frf! ^r {C,E) ^ (C',^'), (2-6) 

where r is a time scale such that W{C',E^ -s— C,E_) is of order unity. Meanwhile, by virtue of 
the ergodicity properties (j2.ip and (j2.3p . the corresponding stationary distribution is merely the 
microcanonical distribution 

P^{C,E) = ^^^^ ^ PnAC,E). (2.7) 
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Ergodicity also entails that, since all microscopic configurations ^'s in the energy level appear 
in the sequence a;, all possible values of x also appear in the coarse-grained sequence w : so 
any mesoscopic state {C,E) can be reached from any other mesoscopic state {C',E^) by a succes- 
sion of elementary transitions, even if they are not involved in an elementary transition (i.e. if 
^{c.E),{C' ,E') = 0); in other words the graph associated to the transition rates W{C',E^ ^ C,E_) 
is connected, or, equivalently, the Markov matrix defined from the transition rates (see definition 
below in (I2.15P ) is irreducible. 

The constraint (|2.5|) imposed by the interaction pattern upon the coarse-grained evolution 
over a period of N time steps entails that the transition rates of the approximated Markov process 
defined in (|2.6p obey two properties. First, 

W{C',E; ^C,E) and W{C,E^C',E;) (2.8) 

are either both = or both 7^ 0. This property may be called microreversibility. Second, if the 
transition rates do not vanish, they obey the equality 

WjCg ^C,E) ^ ndeciC',E') ^ PraciC',E) 

w{c,E^C',E^) n^cc{c,E) PnAc,E)- 

Observe that the arbitrary time scale r has disappeared in this equation. The equality between 
the ratio of transition rates and the ratio of probabilities in the corresponding stationary distri- 
bution is the so-called detailed balance relation. Here the stationary distribution is that of the 
microcanonical ensemble, and we will refer to relation (|2.9p as the microcanonical detailed balance. 

The microcanonical detailed balance (j2.9p for the transition rates W{C',E_' <— C,E_) can be 
derived in another context where the discrete dynamical variables are replaced by continuous meso- 
scopic observables evolving under a Markov process which arises from an underlying microscopic 
dynamics whose Hamiltonian is an even function of momenta (see the derivation in Appendix [C)) . 
In the latter context the fact that the stationary measure is the microcanonical distribution is en- 
forced by the invariance of the Liouville measure in phase space under the Hamiltonian evolution 
(see the derivation of (|C.5p ): the microcanonical detailed balance mainly arises from the invariance 
under time reversal of the trajectories in phase space (see (jCTp ). In our argument for discrete 
variables, the fact that the stationary measure is the microcanonical distribution arises from the 
ergodicity of the microscopic map T (see (|2.ip and (|2.3I1 ): the microcanonical detailed balance 
emerges from the equality between the frequencies of a given transition and the reversed one over 
a period of the microscopic map determined by the total energy, equality which is enforced by the 
star-shaped interaction pattern (see (j2.5p '). 



2.2.3 Further consequence of the interaction pattern 

In the interaction pattern large parts do not interact directly with one another and the energy of a 
single large part is changed in a given transition. We let VlaiEa) be the number of configurations 
in large part a with energy Ea when it is isolated. Then rtdcc{C,E_) — Y[a ^a{Ea), so if {C',E^) is 
obtained from {C,E_) by an energy exchange with bath a which makes C jump to C we have the 
result, with the notation introduced in (|2.2p . 

■f.^^^r "dcc(c',^0 _ Ub^bjE',) _ ngjE',) 

We have used that E^^ — E^ — 6a,b [£{C') ^ ^(C)], so that for 6 7^ a the multiplicity factors are 
unchanged in the transition. 

The latter ratio of microstate numbers can be expressed in terms of the Boltzmann entropies 
when each part a is isolated. The dimensionless Boltzmann entropjH S^{Ea) for the isolated part 
a when its energy is equal to Ea is defined by ^a{Ea) = expS'^(i?a). With these notations, if 
the transition rate W{C' ,E^ \^ C,E_) where C = F^C is nonzero, then the transition rate for the 



The Bolztmann constant is set equal to 1. 
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reversed jump W{C,E C',E^) where C = ¥ JZ' is also non zero (see (|2.8p ) and, by virtue of 
(PH?)) the relation ((^ is reduced to 



The latter formula is the first important stage of the argument. It has already been used in the 
literature, for instance in Ref . |40j . 

We stress that the present argument does not involve any kind of underlying time reversal. 
Here the time reversal symmetry arises only at the statistical level of description represented by 
the Markov evolution ruled by the transition rates. 

Notice also that, as only certain ratios are fixed, different ergodic deterministic microscopic 
dynamics can lead to very different transition rates, a remnant of the fact that the coupling 
between a large part and the small part can take any value a priori. 

Formula (|2.1ip is also a clue to understand a contrario what kind of physical input is needed 
for the homogeneous Markov approximation to be valid. Indeed, why didn't we do the homoge- 
neous Markov approximation directly on the small part ? We could certainly imagine dynamics 
making this a valid choice. However, it is in general incompatible with the pattern of interac- 
tions (see the end of subsubsection I2.1.2p which is the basis of our argument. For instance, if 
in the coarse graining procedure we had retained only the configurations C's of the small part, 
then the corresponding graph introduced in Appendix]^ would have been a cycle instead of a tree 
in the case of a small part made of two spins (see paper II|1S]), and the crucial property (|2.5p 
would have been lost : over a period of N time steps of the microscopic dynamics Ncfi' ^ Ncfi- 
In fact, we may expect, or impose on physical grounds, that fla will be exponentially large in 
the size of the large part a (i.e. its number of degrees of freedom Afa), so that even the ra- 
tio ^a{E'g^)/^a{Ea) = ^a{Ea — [£{C') — £ (C)]) / fla^Ea) will vary significantly over the trajectory, 
meaning that transition probabilities for the small part only cannot be taken to be constant along 
the trajectory: the energies of the large parts are relevant variables. 

2.3 Transient regime when large parts are described in the thermody- 
namic limit 

2.3.1 Large parts in the thermodynamic limit 

We now assume that the large parts are large enough that they are accurately described by a 
thermodynamic limit, which we take at the most naive level. To recall what we mean by that, we 
concentrate on one large part for a while, and suppress the index used to label it. Suppose this 
large part has Af degrees of freedom, and suppose that energies are close to an energy E for which 
the entropy is [E) . That the thermodynamic limit exists means that if one lets M — >■ -|-oo while 
the ratio E/M goes to a finite limit e, there is a differentiable function s^(e) such that the ratio 
S^{E)/N goes to s-^(e). The quantity 



is nothing but the inverse temperature. In that case, as long as ^^E ^ N5E (where 5E is some 
microscopic energy scale) S^{E -\- AS) — S^{E) — >■ /3Ai? when N — > +00. For M large enough, 
the relation S^{E + A£;) - S^{E) /3A£' is a good approximation. 

2.3.2 Transient regime and modified detailed balance (MDB) 

Notice that when transitions occur, which, by the definition of t in (|2.6p . happens typically once 
on the macroscopic time scale, the changes in the energies of the large parts are finite, so that 
over long windows of time evolution, involving many changes in the small part, the relation 



W[C',E^l^C,E) 
W{C,El^C',g) 



^a{Ea) 




(2.11) 




(2.12) 



5f(K)-5f(i^a)~/?a[K-^a] 



(2.13) 
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is not spoiled, where E'^ and Ea are the energies in large part a at any moment within the window. 

In fact the larger the large parts, the longer the time window for which (j2.13p remains valid. 
The relation between the sizes Afa 's of the large parts and of the length of the time window depends 
on the details of T (which still has to fulfill the imposed physical conditions). This relation also 
depends on the values of the energy per degree of freedom in every large part, Ea/Ma, which are 
essentially constant in such a window. 

Because of the ergodicity hypothesis, the largest window (of size comparable to the period 
of T to logarithmic precision), has the property that the energies E^s in the large parts will 
be such that all /3a s are close to each other and the system will be at equilibrium. Indeed, 
inside the largest time window, the system remains in the region of the energy level where the 
energies E^s are the most probable, and in the thermodynamic limit the most probable values 
for the Ea^s in the microcanonical ensemble are the values E*^s which maximize the product 

^a{Ea) under the constraint E — (since the system energies are negligible with respect 

of those of the large parts). The latter maximization condition is equivalent to the equalities 
ds^ / dta{(*a) — ds^ / deii{el) for all pairs of large parts. 

However, if the system starts in a configuration such that the /3a s are distinct, the time window 
over which Ea/Ma and /3a are constant (to a good approximation) will be short with respect to the 
period of the microscopic dynamics, but long enough that (I2.13P still holds for a long time interval. 
Then by putting together the information on the ratio of transition rates in terms of Boltzmann 
entropies (|2.1ip . the transient regime approximation (|2.13p and the energy conservation (|2.2p . we 
get 

WjCE^l^C^E) ^ ^_,„[,(c')-.(c)], (2.14) 
W{C,El^C',g) 

Now the right-hand side depends only on the configurations of the small system, and the param- 
eters f3a& are constants. Letting the large parts get larger and larger while adjusting the physical 
properties adequately, we can ensure that the time over which (j2.14p remains valid gets longer and 
longer, so in the thermodynamic limit for the large parts, the transient regime lasts forever. This 
situation is our main interest in what follows. 

The transition rates in the transient regime correspond to a Markov matrix M defined by 

(C'lMlO J(^'l^l^) "'^^ (2.15) 

\-Ec'(c'|w|c) if c' = c. ^ ' 

As well as the transition rates W{C' ,E^ 1-^ C,E) they must satisfy the three consequences derived 
from the properties of the underlying microscopic deterministic dynamics pointed out in subsub- 
section 12.1.21 namely ergodicity, energy conservation and specific interaction pattern. First, as 
shown in subsubsection 12.2.21 the Markov matrix is irreducible, or in other words the graph as- 
sociated with the transition rates is connected (see ([Lip ). Second, from (j2.8l) the transition rates 
must obey the microscopic reversibility condition (jl.2l) for any couple of configurations (C,C'). 
Third, from 1)2. 14p one gets a constraint obeyed by the ratio of transition rates in the transient 
regime, 

forC'^F.C Ij^-^^^-t^^'^'^-^''^^]- (2.16) 

The latter relation is the so-called modified detailed balance (MDB), which is also referred to in 
the literature as the "generalized detailed balance". 

We stress that, by selecting a time window while taking the thermodynamic limit for the 
large parts, the microcanonical detailed balance (j2.9p is replaced by the modified detailed balance 
(j2.1ip . except in the case of the largest time window where all /3a 's are equal. In the latter case, 
the microcanonical detailed balance (j2.9p is replaced by the canonical detailed balance and the 
statistical time reversal symmetry is preserved. Indeed, the equilibrium thermodynamic regime 
is reached either if the we start from a situation in which Y\a^a{Ea) is close to its maximum 
along the trajectory in the energy level E^ec — E, or if we wait long enough so that fl^ Oa(£'a) 
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becomes close to this maximum. As recalled above, this is true for most of the period of the 
microscopic dynamics, but reaching this situation may however take a huge number of time steps 
if the starting point was far from the maximum. By an argument similar to that used in the 
derivation of (|2.13p . when Yia ^a{Ea) is closed to its maximum and the large parts are considered 
in the thermodynamic limit, all /3a s are equal to the same value /3 and the relative weight of two 
configurations in the microcanonical ensemble, Pmc{C' , E_') / P^^dC , E) given by (12. 7p . is shown to 
tend to exp (— (C) — £{C)]). Then the equilibrium microcanonical distribution PmciC,E_) tends 
to the canonical distribution 

PLiC) ^ (2.17) 

where Z{f3) is the canonical partition function at the inverse temperature /?. Meanwhile, the 
detailed balance relation (j2.9p in the microcanonical equilibrium ensemble for the transition rates 
W{C',E^ <— C,E) becomes a detailed balance relation in the canonical ensemble at the inverse 
temperature /3 of the whole system for the transition rates (C'|W|C), namely 

(CMC)-' ~ pUcV ^ ^ 

The modified detailed balance (|2.16p . valid in transient regimes, differs from the latter detailed 
balance in the canonical ensemble by two features : the various /3a 's of the distinct large parts 
appears in place of the common equilibrium inverse temperature /3, and the stationary distribution 
for the transition rates is not known a priori. 

We conclude this discussion with the following remarks. We have not tried to exhibit explicit 
physical descriptions of the large parts, or explicit formulae for the dynamical map T. Though it 
is not too difficult to give examples for fixed sizes of the large parts, it is harder to get a family of 
such descriptions sharing identical physical properties for varying large part sizes, a feature which 
is crucial to really make sense of the limits we took blindly in our derivation. It is certainly doable, 
but cumbersome, and we have not tried to pursue this idea. Let us note also that in principle, 
taking large parts of increasing sizes can be used to enhance the validity of the approximation 
of the (discrete-time) Markov chain by a (continuous-time) Markov process. As the physics of 
the continuous time limit does not seem to be related to the physics of convergence towards a 
heat bath description we have preferred to keep the discussion separate, taking a continuous-time 
description as starting point. 

2.3.3 Expression of MDB in terms of exchange entropy variation 

Observe that though we have given no detailed analysis of the dynamics or the statistical properties 
of the large parts, their influence on the effective Markov dynamics of the small system enters only 
through the inverse temperatures Pa defined in (j2.12D . So we can consistently assume that each 
large part becomes a thermal bath with its own temperature. The leading term in Sa {Ea) — Sa {Ea) 
is the variation 5Sa" {C -s— C) of the thermodynamic entropy of bath a when it flips the system 
from configuration C to configuration C, 

55r(C'^C)=|^''[^^"^'^] ^'^^"^ (2.19) 
I otherwise. 

Then we have an idealized description of a thermal contact between to heat baths. This is the 
situation on which we concentrate in this paper. 

Since £[¥ aC) — £{C) corresponds to a variation in the energy level populations of the two-spin 
system, it corresponds to the heat Sqa{C' C) received by the two-spin system from the part a 
when it evolves from configuration C to configuration C = F^C, 

iSqaiC ^ C)^£iC')-£iC) if C'^FaC 
|(5ga(C' ^ C) = otherwise. 
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According to the energy conservation relation (|2.2p the variation 5S'^"{C' C) of the thermo- 
dynamic entropy of the bath a when it flips the system from configuration C to configuration C, 
defined in (|2.19l) . reads 

SSl'^iC ^C) = -MqaiC ^ C). (2.21) 

Let us introduce the exchange entropy variation [f| of the system (5exch'S'(C' C) which is 
associated with the heat exchanges with the two thermostats when the two-spin system goes from 
configuration C to configuration C . Thanks to the definition (|2.19p 

'5exch^(C' ^ C) = - ^ SSriC <- C), (2.22) 

a 

namely 

<5cxch5(C' ^ C) = Y,PaSqa{C' ^ C). (2.23) 

a 

Then the modified detailed balance (|2.16p can be rewritten in a form which does not involve 
explicitly the heat bath responsible for the transition from C to C, 

(CMC) _ ^_,_,s(c>^c) 



{CMC) 



^ g-ocxoh^ic ^c;^ (2.24) 



3 Master equation, exchange entropy flow and various en- 
tropy variations 

3.1 Evolution of the probability distribution without MDB (known re- 
sults) 

In this subsection we recall previously known results which are important milestones to our original 
results and which are consequences of the first two properties p.ip and (jl.2p . among the three 
mesoscopic conditions derived from the ergodicity of the underlying conservative deterministic 
dynamics. 

The starting point is that the evolution of the probability P{C; t) that the system is in config- 
uration C at time t is ruled by the master equation 

5](C|W|C')P(C';t)- ^(C'|W|C)P(C;t) = ^(C|M|C')P(C';i), (3-1) 

C'=jtC C'^C c 

where IM is defined in (j2.15|) . Since P{C; t) is to be interpreted as a probability distribution, it has 
to satisfy the positivity and normalization conditions, 

VC P{C;t)>0 and ^P(C;t) = l. (3.2) 

c 

3.1.1 Generic properties 

According to the theory of systems of ordinary differential equations with constant coefficients, 
the solution P{t) of the equation (|3.ip exists and is unique for a given initial function P(io)- The 
Markov matrix IM obeys the property ^, (C|IM|C')P(C'; t) = for any P and this ensures that 
the normalization constraint is preserved under the time evolution, 

^P(C;t = 0) = l ^ Vi>0 ^P(C;t) = l. (3.3) 
c c 

The above property of IM also ensures the existence of at least one stationary solution, but there is 
no argument for every stationary solution to obey the positivity constraint in 1)3. 2p without further 
assumptions. 



Exchange entropy variation is an abbreviation for variation of entropy due to heat exchanges with a bath. 
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3.1.2 Properties arising from the irreducibility condition 

When the transition rates satisfy the irreducibility condition (jl.ip . and if positivity and normal- 
ization p.2p are satisfied at the initial time, the solution of the master equation (|3.ip not only 
meets these conditions for being a probability distribution at any subsequent time, but it even has 
the more stringent property that any configuration has a strictly non- vanishing weight, 

Vt>0 VC P(C;i)>0. (3.4) 

This result can be derived in the framework of the theory of ordinary differential equations with 
constant coefficients 

Moreover the irreducibility condition (|l.ip allows one to build at least formally a stationary 
solution which fulfills the conditions (j3.2|) for being a probability (with the even more stringent 
property p.4p '). This solution is obtained in the framework of graph theory by using the network 
representation of the master equation; the corresponding expression is called Kirchhoff's theorem 
in Ref [52\ 

The irreducibility condition (|l.ip also entails that the stationary solution of the master equa- 
tion is unique, and therefore coincides with the expression given by Kirchhoff's theorem. The 
uniqueness of the stationary solution can be derived either in the framework of matrix theory by 
using the Perron- Frobenius theorem (see for instance [47 1) or in the framework of the theory of 
ordinary differential equations with constant coefficients [52J. 

The latter derivation uses the stability criterion introduced by Schogl [53J. As noticed by 
Schlogl a good candidate for the Liapunov function involved in a stability criterion is the relative 
entropy introduced by Kullback and Leibler [54 in the context of information theory, namely 

S,4P{m,] EE ^P(C;t) In (3.5) 

The relative entropy is well-defined at any time according to (|3.4p . (In the generic case 5*101 [-P|-Po] 
is well defined only if Pq{C) = implies P{C) = 0.) The definition of S'roi[-P|-Po]- with any given 
Pq, entails that 5ici[P|Po] is positive for any P and vanishes only when P is equal to Pq (by virtue 
of the inequality Inx < x — 1 if x > and x ^ 1). Therefore 

5,ci[P|Pst] > if P ^ P.t and 5,ei[Pst|Pst] = 0. (3.6) 

The definition also ensures that S'ioi[P|Po] is convex (i.e. concave upward) for any P, so that 

S^^^Srci[P\Pst] > for any P, (3.7) 

with 



2 ^ dP{C)dP{C') 



5P{C)5P{C') (3.8) 



and SP{C) = P{C) — P{C). Moreover, because of the structure of the master equation p.ip 
combined with the properties Ina; < cc — 1 for any a; > and ^^,(C|[M|C')Pst(C') — 0, the time 
derivative of Src\[P{t)\Pst\ is negative at any time [52], 

dS^^APpm < 0. (3.9) 

The properties p.6p and p.9p define a Liapunov function and ensure that for any initial distri- 
bution P{to) in the vicinity of Pgt limt_j.+oo P{t) = Pst (because the property p.6p ensures that 
i5*-^-'S'rei[P|Pst] > for any P in the vicinity of Pst). With the extra property (j3.7p one can apply 
the stability theorem by Schlogl which states that limt_>4_oo Pit) — Pst for any initial distribution 
P(to)- The interpretation given by Schogl of the stability condition dSrei[P{t)\Pst]/dt < can be 
rephrased as follows. If the observer does not know more than that the system was initially in 
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some unknown state Pq of the stability region, then an unbiased estimate for the state P{t) at 
time t would be Pgt- But if the observer knows Pq by measurements, then his excess knowledge at 
initial time is equal to 5*101 [-Pol^st], and the property dSrci[P{t)\Pst]/dt < reflects the fact that 
the spontaneous development of the states after the last observation can only go such that this 
knowledge does not increase. 

3.2 Microscopic currents 
3.2.1 Definitions 

Our results in the framework of Markovian stochastic dynamics described by a master equation 
can be written in compact forms if we introduce the generic current j{C) for a microscopic variation 
when the system goes out of a given configuration C. Such a current is associated either to the 
variation 0{C') — 0{C) of a configuration observable 0{C) and then 



Jo 



(C)^^(C'|W|C)[0(C')-0(C)], (3.10) 



or more generally to some exchange quantity SK{C' C) such as a heat amount or the variation 
of the reservoir entropies when the system jumps from configuration C ot C, and then 

jsk{C) = Y,iC'MC)dK{C' ^ C). (3.11) 
c 

We include in the definition of an exchange quantity that it obeys the antisymmetry property 
SK{C' ^ C) — —SK{C ^ C). We notice that the above definitions are valid even if the observable 
O (or the variation 5K{C' -s— C)) depends explicitly on time. 

The average of a current j(C) at time t is given by the generic formula for the mean value {0)t 
of a configuration observable 0{C) at time i, namely 

{0)t^Y.^{C)P{C-t). (3.12) 
c 

According to the master equation p.ip the time derivative of the mean value {0)t of a configuration 
observable 0{C) which does not depend explicitly on time is equal to the mean value {io)t of the 
associated current jo{C), 



d{0) 



- = {jo)u (3.13) 



dt 

where jo[C) is the current p.lOp of observable O associated with the stochastic jumps going out 
of configuration C. 



3.2.2 Probabilistic interpretation of microscopic observable currents 

To every observable i.e. to every real-valued function 0{C) on the configuration space, one can 
associate the random process 0{Ct) where Ct denotes the configuration of the system at time t. 
Note that P{C:t) is nothing but P{Ct = C), so that {0)t as defined above could also be written 
{0{Ct)). 

The process associated to the current observable io{C) at time t has a clear probabilistic 
meaning. Without explaining the details, let us just say that the decomposition 

0{Ct) = (o{Ct) - ^ dsjoiCs)^ + ^ dsjoiCs)^ = Mt + Nt, (3.14) 

called the Doob-Meyer decomposition of the process 0{Ct), expresses 0{Ct) as the sum of a 
martingale Mt and a predictible process Nt vanishing at i = 0. Such a decomposition is unique. 
Informally a martingale is a process whose expectation in the future knowning all the past up to 



19 



now is equal to its present value. In particular, the expectation {Mt) is equal to 0. Taking this as 
a fact, one gets immediately p.l3p . Informally again, Nt is predictible because its value at t + dt, 
Nt+dt, is not sensitive to the randomness (i.e. to a possible jump occuring) between t and t + dt. 

Even if 0{Ct) depends only of the configuration at time t, this is not true anymore of Alt and 
Nt which in general depend on the history up to time t. Moreover, the decomposition does not 
behave trivially under nonlinear maps, so that is not the predictible process that appears in 
the Doob-Meyer decomposition of [©(Ct)]^. 



3.2.3 Mean energy time derivative 

As an example of (j3.13L the time derivative of the mean energy {£)t is equal to the mean value of 
the energy current js{C). In the present model, energy variations are only due to heat exchanges 
with the two thermostats according to the conservation rule 

£{C') - £{C) = 6qi{C' ^ C) + dq2{C' ^ C), (3.15) 

where Sqa(C' C) is the heat received from thermal bath a for a jump of configuration as defined 
in (|2.20p . Therefore the energy current jsiC) can be split into two heat currents jsq^ (C) and jsq^ (C) 
received from the thermal baths 1 and 2 respectively. Then the evolution equation p.l3p applied 
to the energy observable £ can be rewritten as 

^ = (ji>. + 02)*, (3.16) 

where ja is a short notation for the instantaneous heat current received from the thermal bath a 
by the system when it leaves configuration C : 

ja{C) EE jsg^iO = J2iC'MC)6qa{C' ^ C) . (3.17) 
c 

Note that, unless (3i — j32, there is no observable Oa for which j5q^{C) would be equal to 
jOai^) with the definition of jo^(C) given in p.lOp . Moreover, under the microscopic reversibility 
condition (|1.2p , if /3i = /32 = /? the modified detailed balance (12.161) becomes the canonical detailed 
balance (12.181) . Then the current associated with any observable Oa (more generally any exchange 
quantity) has a zero mean in the stationary equilibrium state with distribution P^'an ■ Uoa) pf — 0- 



3.2.4 Mean exchange entropy flow 

The heat currents are associated to an exchange entropy variation of the system (see footnote |3] for 
the meaning) according to the relation (12.231) . Similarly to the definition (|3.17p of the microscopic 
heat current ja{C) in terms of the heat amount Sqa{C' C), the microscopic exchange entropy 
current js^^^^^siC) when the system goes out of the configuration C is defined from the exchange 
entropy variation ScxchS{C' C) as 



j5_,5(C) = ^(C'|W|C)(5e.ch5(C' ^ C). (3.18) 



c 

The mean exchange entropy flow is deflned as 

^exch S 



dt 

C,C' 

It can be expressed as the average of the current js^^^hsi^) 



^(C'|W|C)Jexch^(C' ^ C)P{C; t). (3.19) 



= O*e.o.s)t. (3.20) 



20 



The current definitions p.l7p and (j3.18|) together with the relation (j2.23p between the exchange 
entropy variation and lieat transfers imply that 

^^^Mi)t+M2)t. (3.21) 

In the stationary state, the time derivative of the mean energy vanishes, namely d < £ >st /dt — 0, 
where (■ ■ •)st denotes an average with the stationary distribution Pgt- Henceforth, according to 
the evolution equation p.l6p of < £ >st, 

(ji)st + (j2)st = 0. (3.22) 

By inserting the current balance p.22p into the expression (|3.2ip of the mean exchange entropy 
flow, we get 

'^^'^^'^ =-(/?! -/32)(j2)st. (3.23) 

St 



dt 



3.3 Evolution of the Shannon-Gibbs entropy 

3.3.1 Definition of the entropy production rate 

The dimensionless Shannon-Gibbs entropy (where the Boltzmann constant is set equal to 1) is 
defined from the configuration probability distribution P(C; t) as 

S'°[P{t)] ^-Y,P(C-t)\nP{C;t) = -(lnP(t))i. (3.24) 
c 

When the evolution is ruled by the master equation p.ip . the time derivative of S^^{t) takes the 
form 

__ = - g(C'|W|C)P(C; t) In = -(j,„p(,))*, (3.25) 

where we have used the definition (j3.10D of an observable current jo{C;t), also valid in the case 
of an observable which depends explicitly on time. 

As it is done for the phenomenological entropy introduced in the thermodynamic of irreversible 
processes [5J[35] the time derivative of S^'^{t) can be split into two contributions, an exchange (or 
external) part d^^chS/dt arising from exchanges with the external reservoirs, and an internal (or 
irreversible) part dintS^'^ /dt due to the internal irreversible processes in the system, 

JCSG rl , Q rl. 

"cxch*-' "int*J /„ „„x 

dt ^ dt di ■ ^ ' ' 

By virtue of its definition p.l9p . d^xchS/dt is expressed in terms of the exchange entropy variation 
'^exch'5'(C' <— C), associated with a jump of the system from a microscopic configuration C to 
another one C By virtue of the definition p.26p d^^tS^'^/dt is a functional of P{C;t) determined 
from dS^'^/dt given in IfJ^ and dexchS/dt given in IfJJ^ . 

3.3.2 Comparison with the thermodynamics of irreversible processes 

An implicit postulate in the modern literature is that, when the system is out of equilibrium and 
evolves on time scales far smaller than the reservoirs, one can still define some universe entropy, 
which coincides with its statistical thermodynamic expression when the system and the reservoirs 
are at equilibrium, and where in the course of time the equilibrium Gibbs entropy of the system 
is replaced by its instantaneous Shannon-Gibbs entropy. In other words, the out-of-equilibrium 
universe entropy variation is the sum of the variation of the Shannon-Gibbs entropy of the system 
and the variation of the total thermodynamic entropy of the reservoirs, namely 

dt dt dt 
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On the other hand, by definition, the exchange entropy flow dcxchS/dt received by the system from 
the reservoirs is the opposite of the time-derivative of the thermodynamic entropy of the reservoirs 
dexchS/dt = —dS^J^ /dt. As a consequence, by virtue of definition p.26p . 

dSuniv dintS^'^ . , 

- ^^■^^> 

The interpretation of the latter equality is that, since the reservoirs are at thermodynamic equilib- 
rium, the variation rate of the universe entropy, dS\j-a\w/dt, is equal to the production rate of the 
Shannon-Gibbs entropy in the system, namely the internal part di^tS^'^ /dt in the time-derivative 
dS^'^ /dt of the system Shannon-Gibbs entropy. As shown below, if the modified detailed balance 
is obeyed, the universe entropy increases when the system is out-of-equilibrium, as the universe 
entropy increases between two equilibrium states, according to the second principle of thermody- 
namics. 

In the stationary state the time derivative of the Shannon-Gibbs entropy (j3.24p also vanishes, 
dS^'^[Pst\/ dt = 0, and the decomposition p.26p leads to 



dt 



-^cxch 



s 



dt 



(3.29) 



Then according to p. 231) . d^i^tS^^ / dt\st can be written as the entropy production rate introduced 
in the framework of irreversible thermodynamics when there is only one independent mean instan- 
taneous current J (see for instance Ref.|56|). namely in the form 

= J" J, (3.30) 



dt 

where T is the so-called thermodynamic force associated with the heat current J. In the present 
case (j2)st — ~(ji)st and in p. 301) we can make the identification 

J ={32) St and ^ = /3i-/32. (3.31) 

(Another identification might have been J = (ji)st and F = (^2 — Pi-) If T2 > Ti, is positive 
and the positivity of di-atS^^ / dt (settled in next subsubsection when the modified detailed balance 
is satisfied) ensures that the mean current (j2)st received from the heat bath 2 is also positive. 

3.3.3 Entropy production rate under MDB 

When the transition rates obey the modified detailed balance (j2.24p . the microscopic exchange 
entropy current defined in p. 181) can be rewritten as 

Then the exchange entropy flow deflned in p.l9p becomes equal to 

while the entropy production rate given by (I3.25P and p.26p becomes equal to 

The latter expression takes the form of a relative entropy so that di^tS'^'^/dt is positive under 
MDB. 
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We notice that, in the generic case where the modified detailed balance does not necessarily 
holds, Lebowitz and Spohn [1 have introduced the splitting of dS^'^ /dt into CToxch + crint, where 
CTexch[-P(i)] is defined in p.33p and (Ji-n_t[P{i)\ is defined as the symmetrized expression of the 
definition in p.34p . 

<y:AP{t)] = \Y1 [{C'MC)P{C- t) - {CMC')P{C': t)] In fcl^lc^pfc'^ ' ^^'^^^ 

which is obviously positive. Another rewritting for the definition of crint[-P(i)] given in p.34p is 

aiAP{t)]^{jS:..s{t))t with 5i„,5(C'^C)^ln^^J|^|p^. (3.36) 

Then the positivity of (Ji-at[P{t)\ may be viewed as arising from the property : for a: > 0, — Inx > 
1 — X. We notice that jSi^tsiC;t) is not the current introduced in [1]. However, the current 
i<5oxchs(^) is introduced through its expression (|3.32l) there and it is used again, under another 
denomination, in [57) . 



3.3.4 Decomposition of the entropy production rate into two positive contributions 



The microscopic variation 5irAS{C' 
tions as 

<5i„t5(C' ^C) = 



C) defined in p.36p can be decomposed into two contribu- 



In 



P{C'-t) ^ P{C-t) 



In- 



Pst(C) 



where 



(3.37) 



(3.38) 



(C|W|C')Pst(C')' 

By analogy with chemical reaction kinetics (see for instance Ref.[52] and subsection 13.3.51 below) . 
5Ast{C' C) may be viewed as the affinity of the elementary reversible reaction (or phase change) 
C ^ C in the stationary state resulting from all possible reversible pair reactions between all con- 
figurations. We notice that 5{atS{C' ^ C) depends explicitly on time, contrarily to SAst{C' ^ C). 
According to the definitions (j3.10l) and (j3.11l) of the currents associated respectively to an observ- 
able O or to some exchange quantity 5K{C' -s— C), the decomposition p.37p allows to rewrite the 
expression p.36p for aint[P{t)] as 



(^iut.[P{t)] = -(jl„[P(t)/P,t])t + UsAjt- 



(3.39) 



We stress that, by virtue of the already used inequality — In a; > 1—x for a; > and according to the 
stationary condition Ec'7^c(<^l^l<^')^st(C')-Ec'#c(<^'NI<^)-Pst(C) ^ (see (|33])), the microscopic 
current jsA^tiC) which we have introduced is positive. 



jsAAC)>0 for any C. 



(3.40) 



The latter positivity may be viewed as a consequence of the fact that jsA^t i^) is the relative entropy 
(for a given C) of the rate (C'|W|C) with respect to the rate (C'|Wt|C) ee [Pst(C)]"^ {C\VJ\C')Pst{C'). 

According to the master equation p.ip . the first contribution in the r.h.s. of p.39[) is equal to 
the opposite of the time derivative of the relative entropy of the probability distribution P{t) with 
respect to the stationary solution defined in p.Sp . Therefore the expression (j3.36p of the entropy 
production rate (Tint[P(t)] can be split into two contributions, both of which are positive, 



CTint[P(<)] 



d^,el[P(^)|Pst 

dt 



+ (jsAjt- 



(3.41) 



The positivity of the first term has played a role in the discussion of the uniqueness of the stationary 
state (see Ea. p.9p ) and the positivity of the second term arises from the positivity of the current 
pointed out in p.40p . 
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For the sake of completeness, we notice that, when the transition rates depend on time the 
master equation p.ip remains unchanged, so that the decomposition of the time-derivative of 
the Shannon-Gibbs entropy into CToxch + crint as well as the decomposition (j3.39p of (Tint remain 
unchanged, except that Pgt is replaced by a function of time, namely the probability distribution 
which is the zero eigenvector of the Markov matrix at a given time. Then the splitting (j3.39p of a-mt 
is quite analogous to that introduced in [39 for an evolution where both the bath temperatures 
and the energy levels may change with time. The so-called non-adiabatic term corresponds to the 
opposite of the time-derivative of the relative entropy Si-c\[P{t)\Pst] and the so-called adiabatic 
term corresponds to the average of the stationary affinity current. 

3.3.5 Comparison with NESS characterization from graph theory 

The r.h.s. of the master equation p.ip can be rewritten as a sum of probability currents between 
configurations, 

^^^=^J[P](C',C;0, (3.42) 

C,C' 

where 

J[P](C',C;t) = (C'|W|C)P(C;t)-(C|W|C')P(C';t). (3.43) 

The probability current J[p](C,C;i) has the form of the chemical reaction rate associated with 
the reversible reaction (or phase change) C ^ C with the species concentrations replaced by the 
configuration probabilities P{C) and P{C') and the reaction rate constant for C C replaced by 
the transition rate (C'|W|C). The entropy production rate a{-a_t[P{t)\, introduced in Ref.[lJ in the 
generic case (namely when the MDB is not necessarily satisfied) by the definition p.35p . can be 
rewritten (with notations similar to those of Ref.[S2]) as 

cTint[P{t)\ - i^M[p](C',C;i)J[p](C',C;i), (3.44) 

C,C' 

where the dimensionless afhnity 5A\^pT^{C',C\t) of an oriented pair (already introduced in the case 
of the stationary distribution Pst in (j3.38p with a slightly different notation) is defined as 

M[p,(C',C;t).ln#^^)^. (3.45) 

'-^'^ ' ' ' {c\y\C')p{C';t) ^ ' 

Indeed, in irreversible thermodynamics of chemical reactions at a temperature T fixed by a ther- 
mostat (see for instance [S^), if -P(C) denotes the ratio between the concentration of species C 
and the sum of all species concentrations, the affinity of the reaction C C is equal to k^T times 
an expression similar to 5A[p](C,C; t), with the concentration of species C (C) in place of P{C;t) 
{P{C'\ t)) and the reaction rate constant for C ^ C in place of the transition rate (C'|W|C). 

The previous rewritings are convenient to handle the master equation in the framework of 
network theory where the master equation is represented by a graph G as follows. Each vertex of 
the graph corresponds to a given configuration C and there exists an edge between two vertices if 
at least one of the transition rates (C'|W|C) or (C|W|C') does not vanish. Moreover an arbitrary 
orientation is chosen for every edge in the graph G so that the transition rate (C|W|C) (or (C|W|C')) 
can be shortly referred to as the transition rate along the edge in either the positive or the negative 
sense. 

From a connected graph G one can define several possible fundamental sets of Nc circuits (or 
closed paths) on the graph. The number Nc of circuits is only determined by the edge number 
and the vertex number iV„ through the relation Nc = Ne—Ny+\. (The final results are independent 
of the specific fundamental set used in intermediate algebraic calculations.) A fundamental set is 
built from one the various possible maximal (or spanning) trees which arc defined by removing 
Nc edges of G. For a given maximal tree T{G) the corresponding removed edges are called chords 
and indexed by a = 1, ... , Nc- For each a the circuit Ga is obtained by first considering the graph 
made by adding the chord a to T(G) and then removing all edges which are not part of the circuit 
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closed by the insertion of the chord a. A cycle Cq, is associated with every circuit Cq by choosing 
an arbitrary orientation to go around the circuit, so "cycle" is a synonymous for "oriented circuit". 

The affinity associated to a cycle Ca is defined as an algebraic sum of all edge affinities 

which is calculated as follows: each edge affinity, whose expression is given by p.45p for a positive 
orientation from C to C in the graph G, is multiplied by the sign of the relative orientation of the 
edge in the cycle Cq and in the graph G. Because of the cyclic structure of Ca the affinity A(Ca) 
does not depend explicitly on the configuration probability distribution P and it is determined 
only from the transition rates (C'|W|C). Indeed, if N{Ca) is the number of configurations involved 
in the cycle Cq,, and if the configurations are labeled with indices increasing with one unit when 
one goes from one configuration to the next one in the sense chosen for the orientation of the cycle 
Cq, then 

with the notational convention C^j.^^ = Ci. The probability current J[p^{Ca\t) associated to 
the cycle is defined as the probability current in the chord a in the sense of the cycle orientation : 
it is given by definition (j3.43p where C and C' are respectively the initial and final configurations 
in the sense of the cycle orientation. 

As shown in Ref.|52|. the stationary state has the following properties. First, the vanishing 
of all cycle affinities A{Ca) is equivalent to the the vanishing of all cycle probability currents 
J[p^j](Cq) in the stationary state, 

Va A(Cq)=0 ^ Va J[p^^] (Cq) = 0. (3.47) 

Second, the vanishing of all stationary cycle probability currents J[p^j](Cq) is equivalent to the 
fact that the stationary state obeys the detailed balance condition, namely 

Vo J,,.,(C,).0 « V(C.C') (3,48) 

Schnakenberg specifies that it is in fact a "complete detailed balance" condition in the sense that 
if there exist several kinds of independent transitions between two configurations C and C, the 
detailed balance must be satisfied by every kind of transition. Since equilibrium is characterized 
on the mesoscopic level by the detailed balance (see the derivation of (j2.18p ) , the properties (13.47P 
and p.48p entails that the equilibrium is characterized by either the vanishing of all stationary 
cycle currents or the vanishing of all cycle affinities. Moreover, the entropy production rate (j3.44p 
in the stationary state reads 

aint[Pst] = E ^(Cq)) J[P.,](Cq), (3.49) 

a=l 

where A{Ca) is given directly in terms of the transition rates by the formula (j3.46p . By virtue 
of the definitions (j3.33p and (I3.34p . (Jexch[-P(i)] + <^int[P{t)] = dS^° /dt and in the stationary state 

When the MDB is satisfied, according to (13.331) . fTnyrhfPatl = dexchS'/iiiLt so that (Tint f^'stl — 

MDB MDB 

—dcxchS/dt\st and p.49p becomes 

'^-'^^ -£a(Cq))J[p,,](Cq). (3.50) 



dt 



, MDB 

St Q = l 



Meanwhile, by virtue of (|1.3p the affinity of a cycle A{Ca) given by (|3.46p becomes equal to 

^(^") .,7^0 - y. ^c.chS{C,+l ^ C,;), (3.51) 
A'lDB ' * 
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where the latter expression depends on the thermodynamic parameters of the reservoirs and on the 
quanta of microscopic quantities which the system exchanges with the reservoirs. In the generic 
case there may be several cycles corresponding to the same current between two reservoirs and 
there is no straightforward correspondence between the expression of — (iexch'S'/(it|st given by (j3.50p 
in terms of the A(Cq,)'s and J[p^j](Cq)'s and the expression of —dcxchS/dt\st given by the entropy 
production rate for the phenomenological entropy S in thermodynamics of irreversible processes 
^dcxchS/dt\st = (iint'S'/dtlst — '^■y J^-yJ^j whcrc the J*'s are independent macroscopic currents. 

However in the case of the simple solvable model considered in paper II , where the thermal 
contact between two thermostats is settled by a two-spin system, only one cycle is involved and 
one can make the correspondence between, on the one hand, the affinity A(C) of the cycle and the 
stationary current J[Pgt](C) which goes through it, and, on the other hand, the thermodynamic 
force and the mean heat current J. Indeed, in this model, the graph associated to the master 
equation is itself a cycle, which can be orientated to read 

(+,+) ^ (-,+) 

t ; . (3.52) 

(+,-) ^ (-,-) 

The cycle may be rewritten as Ci — C2 — >• C3 — > C4 — > Ci where the configurations are labeled 
in the positive sense of the cycle orientation. Then, since the model obeys the MDB, the affinity 
of the cycle is given by p.5ip . and by virtue of the definitions (|2.20p and (|2.23p it reads A{C) = 
— /3igi(C') — ^2^2(0), where qa{C) is the heat received from the thermostat a when the system 
configurations perform the cycle once in the positive sense. Moreover, according to the energy 
conservation law, qi(C) + q2{C) is equal to the energy difference between the final and initial states 
when the cycle is performed once : this difference vanishes for a cycle so that 

A(C) - (/3i-/32)g2(C). (3.53) 

On the other hand, since the graph is exactly a cycle, the current along an edge defined in (j3.43p 
has the same value for all edges in the stationary state (because, by virtue of p.42p . the stationary 
condition dP{C\t) / dt\st = is equivalent to Kirchoff's current law at every vertex of the graph). 
As a consequence, the stationary current associated to the cycle reads 

J[P^,](C) = (C,+i|W|C,)Pst(C.) - (C,|W|C,+i)Pst(C,+i), (3.54) 

where i is any label in {1,2,3,4}. The mean exchange entropy fiow given by p.50p reads 
deyicYiS / dt\st = —{Pi — P2)q2{C)J[p^^]{C). Comparison with the expression dcxchS/dt — —J-'J 
of irreversible processes thermodynamics (see p.29l) - p.30p - (|3.3ip ). leads to the following identifi- 
cation of the macroscopic heat current J = (j2)st received from the heat bath 2 (and given to the 
heat bath 1), 

J = g2(C)J[p^,](C), (3.55) 
while the thermodynamic force J- — Pi — (32 is to be identified with 

J-=^. (3.56) 
92 (C) 

The probabilistic interpretation of the affinity A{C) is given in paper II |45| . 



4 Exchange entropy variation and symmetries at finite time 
under MDB 

4.1 Exchange entropy variation for a history 

For a history Jiist where the system starts in configuration Co at time Iq = and ends in configu- 
ration C/ at time t after going through successive configurations Cq, Ci,. . . , Cjv = C/, the exchange 
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entropy variation AcxchS[Hist] corresponding to the history is defined from the heat amounts 
Qi[Hist] and Q2['Hist] received from the thermal baths as 

N-l 

Aexch5[Hlst] = /3lQl[H^.S^] + ^2Q2[nist] with Qalnist] = Sqa{C^+l ^ C,). (4.1) 

The expectation value of /S.^y^dnS^Hist] with respect to the measure over all possible histories 
starting from configurations distributed according to some initial probability distribution (see 
appendix [D| is equal to the time integral of the mean exchange entropy current calculated with 
the instantaneous configuration probability distribution, 

(Aexch5) = dt\js^^^,s)t' = dt'^^. (4.2) 
The second equality arises from p.20p . 



4.2 MDB and symmetry between time-reversed histories 

Let T be the time reversal operator for histories. If Hist is a history that starts at time tg = 
in Co and ends at time t in Cf after N jumps from Ci-i to Ci at time Ti, JHist is a history that 
starts at time to = in C/ and ends at Co at time t after N jumps from Cl_i to C^' at time T/ with 
Cj' = Cn-i and T[ = t — T/v-j+i , namely 

Hist: Co at to = Co — ^ Ci • • • Cat-i — ^ C/ (4-3) 

JHist : Cfaito — O Cf — ^ Cw-i • • • Ci Co. 

From the definition of the measure dPcg^Cj over histories starting in configuration Co and ending 
in configuration C/ (see Appendix [D)) . 

dPcf,Co [Hist] ^ ^1 (C,+i|W|C,) 
dPcoA[™*s^] fi (C.|W|C,+i)- 

When the transition rates obey the modified detailed balance (|2.24p written in terms of 
<^oxch'S'(C' ^ C), the exchange entropy variation for the history, defined in (j4.ip . can be rewritten 

^ N 

Wh^[Hz.t] ^ -Inn STJ^^v (4-5) 

MDB fj-^ (Cj|W|C,+i) 



and equation (|4.4I) can be rewritten as 



dPcffio [Hist] _ „_A,x,„s[Wst] 



dPca.Cf [TOist] MDB 

We stress that, according to (j4.5p . when the MDB is satisfied the expression of the exchange 
entropy variation for a history defined in (14. ip coincides with the opposite of the "action func- 
tional" introduced by Lebowitz and Spohn in Ref. [T] in the generic case where the MDB does not 
necessarily hold. 



4.3 Symmetry between time-reversed evolutions with fixed heat amounts 

The probability P (C/|Qi, Q2, ^|Co) that the system has evolved from the configuration Co at to = 
to the configuration C/ at t while receiving the heat amounts Qi and Q2 from the thermostats 1 
and 2 reads 

P(C/|Qi,Q2;t|Co)= / dPc,fio[Hist]5{Qi[Hist]~Qi)5{Q2[nist]~Q2), (4.7) 
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where JdPcffig denotes the "summation" over the histories from Cq to Cf. The time-reversal 
symmetry property (|4.6p for the history measure dPcf.Co [hList] imphes the following relation be- 
tween probabilities of forward and backward evolutions where initial and final configurations are 
exchanged (and the heat amounts are changed into their opposite values), 

P{Cf\Qi, Q2;^|Co) ^ „-A„,,hS(Qi,Q2) (A o^ 

P{Ca\-Qu~Q2-ACf) ' 

with the definition 

Aexch5(Ql,Q2) = /3lQl+/32Q2. (4.9) 

An analogous relation for Aexch-S' in place of (Qi,Q2) is derived in |59J in the case where the 
microscopic dynamics of the heat baths are assumed to be Hamiltonian. 

4.4 Symmetries in protocols starting from an equilibrium state 

We consider a protocol where the system is prepared in an equilibrium state at the inverse tem- 
perature /?o and then put in thermal contact with the two thermostats at the inverse temperatures 
/?! and /32 respectively at time to — Q. Such a protocol is a transposed version of the Jarzynski 
procedure [30J where the system has been prepared in an equilibrium state at the initial time 
and then is submitted to a variation of some control parameter which drives the system out of 
equilibrium by injecting some work. In the present protocol, work is replaced by the heat flux 
that goes through the system from one thermal bath to the other, and the system evolution is a 
relaxation from an equilibrium state to a stationary non-equilibrium state. 

The initial equilibrium distribution at the inverse temperature /3o is the canonical distribution 
(l2Tf|) . Z(/3o) cancels in the ratio pA'„(Co)/Pc1n(C/) and 



l^fTTT = /3o [^(C/) - f(Co)] - /?o(Qi + Q2), (4.10) 



where the last equality is enforced by energy conservation. Then the time-reversal symmetry (|4.8p 
and the specific form (|iJU)) for ln[P/^o„(Co)/Pcan(C/)] imply that 



P(C/|Qi,g2;t|Co)Pf,°„(Co) ^ ^-A— '^0^,(2^,2, 
P{Co\~Qi,-Q2-ACf)Ptn{Cf) 



where A°™^'^°iS'(Qi, Q2) is an excess exchange entropy variation defined as 

^Z^^^'SiQi, Q2) = Ae.ch5(Qi, Q2) - /3o(Qi + Q2) = iPi - MQi + {132 - MQ2. (4.12) 

^oxch''°'^(2i' 22) does not depend explicitly on the final and initial configurations and is only a 
function of the heat amounts received from the thermal baths. 

As a consequence, the measurable joint distribution Pp^o (Qi, Q2; for the heat amounts Qi 
and Q2 received between to = and t when the initial configuration of the system is distributed 
according to P^°^^, which is defined as Ppft,^ (Qi, Q2; i) = Eco.C/ ^ (^/l 2i' 22, t|Co) Ppftj^ (Co), 
satisfies the identity 

Pp,AQl,Q2:t) A— ■^OS(Q,.Q.) u 13) 

Pp.oJ-Qi,-Q2;t) " ■ ^ ' 

Subsequently the measurable quantity ^^^^i^'' S{Qi, Q2), with the distribution probability 

^Pcl^^ (^excf ^) = EQ„s.'5(A:::^f''5-(/3i-/3o)Qi-(/32-/3o)S2)Pp.c, (Qi,Q2;t) obeys 
the symmetry relation at any finite time, which may be referred to as a detailed fluctuation 



relation. 



(A:xcf"^) 
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The latter relation itself entails the identity, which may be referred to as an integral fluctuation 
relation, 

(eAe-^'^««)^,„^ ^ 1. (4.15) 

To our knowledge these two relations have not appeared explicitly in the literature. However 
they may be seen as another application of the argument first exhibited by Crooks |36j and then 
Seifert [3H1 [S| for the entropy production along a stochastic trajectory when the system is in 
thermal contact with only one heat bath and is driven out of equilibrium by a time-dependent 
external parameter. (In Crooks' argument the initial configurations for the forward and backward 
evolutions are distributed with different equilibrium probabilities, P^°^ and Pcln, whereas forward 
and backward evolutions with the same initial distribution had already been considered in j60ll61| ). 

4.5 Symmetries in protocols starting from a stationary state with a 
canonical distribution 

For some systems, such as the two-spin model studied in paper II , the stationary distribution when 
the thermostats are at the inverse temperatures /3i and (32 proves to be a canonical distribution 
at the effective inverse temperature /3^(/?i, /32). 

When the system is prepared in a stationary state between two heat baths at the inverse 
temperatures Pi and and then put in thermal contact with two thermostats at the inverse 
temperatures /3i and at time to = 0, the protocol describes the relaxation from a given stationary 
state corresponding to {PitP^.) ^ another stationary state corresponding to (/3i,/32)- When the 
initial stationary state has the canonical distribution at the effective inverse temperature j3l — 
/3j,(/3°, /32)j the argument of the previous section can be repeated and the equalities (|4.14p and 
(|i:T31) still hold with /3o replaced by /?° and l^'^^l^" S replaced by 

^ZlfsiQi, Q2) = - PDQi + {(32 l3l)Q2. (4.16) 

When the system is already in the stationary state corresponding to the inverse temperatures 
Pi and P2 at time = 0, the equalities and ^jU^ for A.im'^°S still hold with ^^,{^1,^2) 

in place of Po : 

V / ^ e-A_h 4.17 

where the subscript st of the probability is a reminder of the fact that the initial configurations 
are distributed according to the stationary measure, which is equal to P^^^ in the present case. 
Another detailed fluctuation relation involving the forward histories for the original dynamics and 
the backward histories for the dual reversed dynamics is derived in |62j for the case where the 
external parameters also vary during the time interval Jtoi^]; these considerations are out of the 
scope of the present paper. 



5 Long-time symmetries : non-equilibrium steady state with 
MDB 

5.1 Fluctuation relations 

5.1.1 Fluctuation relation for the cumulated exchange entropy variation 

As recalled with some details in section |3l if the Markov matrix is irreducible and the system has 
a finite number of configurations, then the Perron-Frobenius theorem entails that there exists a 
single stationary state Pst and every configuration C has a non-zero probability Pst(C). Let us 
call P^™ and Pgf^^ the minimum and maximum values taken by Pst- Since Pst {Qi, Q2',t) = 
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Eco.c, ^(C/|Qi, Q2;t|Co)Pst(Co), and < < g^, the thne-reversal symmetry ^ 

for F(C/|Qi,Q2;i|Co) entails that 



max 
st 



The distribution probability for the exchange entropy variation AoxchS' can be determined from 
measurements of heat amounts through the relation 

Pst(Aexch5;0 - J2 '5(Acxch5-/3lQl-/32Q2)Pst(Ql,Q2;t). (5.2) 
Qi,Q2 

The inequalities (j5.ip imply that 



Ps?'" ^ Pst (Aexch'S'; t) ^ Pg™'"'" ('r o^ 

P,r" " Pst (-Acxch^; i) e-^-chS - pmi„ • ^ • J 

In the long-time limit the system reaches its non-equilibrium stationary state exponentially fast, 
so the existence and value of a large deviation function for the cumulated current = AexchS/t 
are not expected to depend on the initial distribution. Appendix |E] contains several definitions of 
large deviation functions. Subsection IE. 31 contains a proof that, whatever the definition, a relation 
like (|5.3p implies that, if AexchS' has a large deviation function fA^^chS{J) ~ under P^t but then 
also under any initial probability distribution - then 

/a_,s(^) - /a_„s(-:^) = ^J- (5.4) 

Our derivation of this fluctuation relation is close to the argument given in Ref. |41| for the large 
deviation of the cumulated heat current (see (IS.DI) ). It relies on the MDB obeyed by the transition 
rates. Since the opposite of the exchange entropy variation is the specific form which the Lebowitz- 
Spohn action functional takes in the presence of MDB, the fluctuation relation (|5.4p is in fact a 
special case of the fluctuation relation satisfled by the action functional for a system with a finite 
number of configurations under the assumption (jl.2l) of reversibility for configuration jumps and 
the assumption (jl.ip that the Markov matrix is irreducible, without the extra assumption of MDB 

5.1.2 Constraints from the bound upon the system energy 

In a system with a finite number of configurations, Qi + Q2 = £{Cf) ^ is bounded and this 

entails several properties upon the large deviation functions of the cumulated currents J7i = Qi/t 
and J2 = QaA- 

When Qi + Q2 is bounded, the consequences for the cumulated heat currents are conveniently 
investigated if one considers the couple of variables (Qf , Q2) where Qf = ~Qi is the heat amount 
dissipated towards the thermal bath 1. The fact that the difference Qf — Q2 is bounded means 
that there exists some (time-independent) cosntant A/ > such that 

\Qt-Q2\<M. (5.5) 

The first straightforward consequence upon the cumulated heat currents J'f = Qi/t and J'2 = 
Q2/t is that limt^+oo(Ji'') = limt^+oo(J^2) = J, namely 

hm ^ . hm ^ = J. (5.6) 

t— >-+oo t t 

The second consequence of the fact that Qf — Q2 is sub-extensive is that, according to (jE.30p . 
Qf and Q2 have the same large deviation function, /gd {J') — Jq,^ {J) , namely 

fQAJ)^fQA-J)- (5.7) 



30 



Similarly the difference between AcxchS' and — — /32)Q2, which is equal to /3i[£(C/) — £{Cf)], 
is bounded. A fortiori Aexch<S' + — P2)Q2 is sub-extensive, so that according to (jE.30p 
fA^^^^siJ) = f-{f3i-/32)Q2iJ')^ namely 

/a_,5(J) = /q. f--T^V (5.8) 



/3i-/32 

As a consequence, the fact that the exchange entropy variation Aexch'5' obeys the fluctuation 
relation (j5.4p is equivalent to the fact that the the cumulated heat current received from the heat 
bath 2 obeys the fluctuation relation 

fQAJ)~fQA~J)^{l3i-l32)J. (5.9) 

The latter equation is the long-time limit of the relation first exhibited by Jarzynski and Wojcik 
[42] for the thermal contact between two Hamiltonian systems initially prepared at different inverse 
temperatures /3i and /32 and set in contact at time Iq — 0, with the implicit assumption that the 
energy variation of the system which ensures the thermal contact is negligible with respect to the 
heat quantity which goes through it. The fluctuation relation (|5.9p is also derived in the framework 
of the master equation approach in Refs.[40, 41 1. 

We notice that, since the function ^^^^h^" S{Qi, Q2) defined in (|4.12p is equal to Acxch5' plus 
a term — /3o(Qi + Q2) which is bounded by virtue of energy conservation, the large deviation 
functions for A|;^^j*j'^''S' and Acxch<5' coincide (see (jE.SOp ). As a consequence, (j5.4l) entails that 
^exch''°(2i' 22) obeys the fiuctuation relation f ....... Poi J) - oxc.,fio (- J') = - J. 

cxch cxch 

The relation (15. 9p can be written in a more generic form by the following argument. The fact 
that the difference between Aoxch<S' and — (/3i — /?2)Q2 is bounded yields the following relation 
between the infinite-time mean values of the corresponding cumulated currents, 

lim i^^^^^ = -(A-/?2) hm i^aM. (5.10) 

t— >-+oo t t— J-+00 t 

Since the infinite-time limit of the mean value of a cumulated current J7t measured during the 
interval [0, t] is equal to the mean value of the corresponding instantaneous current in the stationary 
state, namely 

lim iJt) = (j%t = J, (5.11) 

we get limt^+oo(AoxchS')/i = (js^^^t^ s) st = rfoxch£M|st as well as limt^+oo{Q2) /t = (j2)st (with 
the instantaneous current definitions (I3.18P and (I3.17P respectively). With these identifications the 
comparison of (|5.10p with the property p.4p (which exhibits the analogy with the thermodynamics 
of irreversible processes) shows that Pi — [32 in (|5.10p is to be interpreted as the thermodynamic 
force T. Therefore the fact that the fluctuation relation (|5.9p arises from the boundedness of the 
difference between Aoxch<S' and — (/3i — /32)Q2, as the relation between the mean values (|5.10p does, 
implies that the fluctuation relation (|5.9p for Jq^ is a special case of the more generic fluctuation 
relation 

f{J;T)-f{-J:T)=TJ, (5.12) 

where is the thermodynamic force which appears in the stationary exchange entropy flow 
dcxchS/dt\st = —TJ. 



5.1.3 The Gartner-Ellis theorem and some of its consequences 

As exemplifled in the previous subsubsection, large deviation functions may depend on auxiliary 
parameters. For certain questions, they are simply spectators. This is the case in the following 
discussion, so we do not mention possible auxiliary parameters explicitly. However, we nevertheless 
write all derivatives as partial derivatives. 

In this short subsubsection, we introduce an important tool to study the existence and prop- 
erties of a large deviation function f{J): the infinite-time limit of the generating function for the 
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cumulaiits of Xt = tJt per unit time, namely a{X) = limt_i.+oo(l/^) ln(e^"^'). This fmiction, if it 
exists, is automatically convex. 

According to a simplified version of the Gartner- Ellis theorem (see e.g. the review for physicists 
or the mathematical point of view [M|)j if exists and is difFerentiable for all A in R. then 
the large deviation function / of the current J exists and it can be calculated as the Legendre- 
Fenchel transform of a(X), namely, with the signs chosen in the definitions used in the present 
paper, 

/(J) = min{a(A) - A^}. (5.13) 

AG [R 

If a(A) is stricly convex and continuously differentiable, then for each JT" the minimum is 
achieved for a single value of A, which is the unique solution \c oi ^ = J i.e. the Legendre- 
Fenchel transform reduces to the usual Legendre transform, and the duality relation, Xc{J) — ~§j 
holds, i.e. 

= J. (5.14) 

A= 



dX 



If moreover a(A) is differentiable twice, taking the derivative of this relation with respect to 
one obtains 

92/ d^a 



{dJY {dXf 



-1. 



(5.15) 



\=K{J) 



From the fundamental properties of any large deviation function. f{J^) is maximum aX J = J 
defined in (|5.1ip . so by construction Ac(J) = 0. But derivatives of q;(A) at A = are related to 
cumulants of Xt at large t. For instance 



{dxy 



A=0 



lim 

t— !- + 00 



t 1st 



{Xt)l 



and one gets: 



{dJf 



lim 



(X2 



t /St 



t/st 



(5.16) 



(5.17) 



We now apply these considerations to linear response. 



5.1.4 Linear response far from equilibrium 

In the case at hand, the large deviation function / depends on other parameters. The one relevant 
for the discussion of linear response is the thermodynamic force and we write f{J\J-). In 
general / depends on other variables that we do not mention explicitly, and which are supposed 
to be kept constant whenever a partial derivative is taken in the sequel. 
We start from the property recalled in the previous subsubsection: 



dj 

Taking the derivative with respect to F leads to 



= 0. 



J=.J 



dT [dJf 



5V(J;-^) 



dTdJ 



(5.18) 



(5.19) 



J=J 



J=J 



According to (|5.17p 
the infinite-time limit, and we can rewrite the equality (|5.19p as 

dJ _d^f{J-F) ^, {Xf 



can be expressed in terms of the second cumulant of Xt in 



dTdJ 



J=J 



X lim 



{Xt)l 



t 



(5.20) 



This is the generic expression of the linear response in the non-equilibrium state far from equilib- 
rium. In the case of thermal contact Xt is the cumulated heat Q2 (t) received from the heat bath 
2. 
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5.1.5 Green-Kubo relation as a consequence of the fluctuation relation for the large 
deviation function 



When the large deviation function obeys the fluctuation relation (|5.12p arising from the MDB, 
successive partial derivatives of the fluctuation relation entail that 



d\f{J-T) , d\f{J'-T) 



dTdJ 



dFdJ' 



1. 



(5.21) 



J'=-J 



Then the relation ((OT|) for J = yields 
= is in fact the equilibrium state, 



dTdJ 



dJ_ 



- X hm 

2 t-5. + CX) 



— i and, since the stationary state with 



oq 



t 



(5.22) 



In the following the latter fluctuation-dissipation relation will be referred to as the Green-Kubo 
relation. The precise terminology has been pointed out in Ref.|51|. When the equilibrium fluc- 
tuations are written in terms of a second-order cumulant, namely the mean value of the squared 
Helfand moment Xt — (Xt), as it is the case for Einstein relation, the fluctuation-dissipation rela- 
tions are called Einstein- Helfand formulae |13l I65| . When the equilibrium fluctuations are written 
in terms of the time-correlation function of the instantaneous current, they are are known as 
the Green-Kubo relations |66l [67l [68l [69] or the Yamamoto-Zwanzig formulae in the context of 
chemical relations [7D1 [7T] . 

The Green-Kubo relation (|5.22p can be rephrased in terms of the Onsager coefhcient L defined 
in the framework of thermodynamics of irreversible processes near equilibrium as 



L = 



dJ_ 
dT 



(5.23) 



where the last equality is valid when there is only one nonzero mean current in the stationary 
state, in which case this single current J vanishes as T goes to zero. 

In the case of thermal contact, the limit — ^ in (I5.23P can be stated more precisely ac- 
cording to the following argument. If /3i = /32 the stationary state is the equilibrium state 
and J{(3i,/32) becomes J{/3,/3) = (j2)oq = (see the comment after p. 171) ). As a consequence 
J(/3 -|- d/3, /3 -|- d(3) — J(/3, /3) = for any infinitesimal d/3, namely 



so that 



dJ 

J(/3i,/32) 



3i=/32=/9 



dJ 

dp2 



/3i=/32=/; 



/3i,/32 



(5.24) 



(5.25) 



3i=/32=/3 



independently of the way /3i and ^2 go to /3. Subsequently the linear response coefficient L defined 
in (|5.23p also reads 

dJ 



L = 



dpi 



while the Green-Kubo relation (|5.22p can also be stated as 



J(/?l,/32 

mT-^P Pi - P2 



lim 



— lim ■ 
2 



QiW)fq-((Q2(t)) 



t 



(5.26) 



(5.27) 



where J(/3i, (32) = (j2)st ^'^'^ the limit is independent of the way P1—P2 vanishes. Eventually in 
the case of thermal contact the Green-Kubo relation relates the proportionality coefficient between 
the stationary heat current and the difference between the bath inverse temperatures when the 
system is weakly out of equilibrium to the fluctuations of the heat amount received from one 
thermal bath in the equilibrium situation where both thermostats are at the same temperature. 
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5.2 Symmetry of the generating function for cumulants per unit time 
under MDB 



Contrarily to the case of the Green-Kubo relation (|5.22p between the mean current and the equi- 
Hbrium infinite-time second cumulant per unit time, relations between higher cumulants per unit 
time cannot be readily obtained from the fluctuation relation (j5.12p . In order to obtain such 
relations one has to resort to the cumulant generating function. 



5.2.1 Infinite-time cumulants per unit time for Qi and Q2 



The fcth cumulant Kq for the heat amount Qa received from the bath a or the joint cumulant 
'^S ''sb heat amounts Qa and Qb can be computed from the characteristic function 



,AiQi(t)+A2S2(t) 



) = E E E E ^ (C/ 1 2i > 22 ; i |Co) P(Co , io = 0) 



(5.28) 



Si Q2 

through the following derivatives 



Cf Co 



jp] _ £)Pln(e^iSi(*)+A2S2(t)) 



4 W = 



,.[P^9] 



Ai=A2=0 



9Ag 

gp+q in(e^i (i)+A2 QaCt)^ 



for a = {1,2} 



(5.29) 



(5.30) 



Ai=A2=0 



For a Markov process, the leading long-time behaviors of these cumulants are proportional 
to the time t elapsed from the beginning of the measurements. The asymptotic behavior of the 
cumulants per unit time are given by the derivatives of 



ai2(Ai,A2)= lim 1 \n{e^^Q^^')+^-Q^W) 
with respect to Ai and A2 at Ai = A2 = 0. In other words, 



lim 



''gi,g2 
t 



Qp+q 



ai,2 



(5.31) 



(5.32) 



Ai=A2=0 



Similarly 



where 



Jp] 

hm ^ 
t 



aa{X) 



dPaa 



dXl 



for a = {1,2} 



Aa=0 



lim -ln(e^2»(*') 
t 



(5.33) 



(5.34) 



The generating function for infinite-time cumulants per unit time, cka(A), may also be referred to 
as the scaled cumulant generating function. 

We notice that, since for any cumulant limt^ooKgj^/t is finite, under some technical con- 
ditions, the probability distribution of the variable [Q2{t) — {Qiit))] / \/i becomes Gaussian in 
the long-time limit. Indeed the logarithm of the characteristic function for the variable l2(t) = 

[Q2(t) - (22(0)] /Vi reads ln(e^^2(*)) = EiS(1/pO (A/Vt)^ Kq, (0- ^ the sum and the t +00 
limit can be interchanged, ln(e^^2(*^) becomes proportional to A^ in the limit where t goes to 
infinity : only the second cumulant of Y2(t) survives in the long-time limit. 
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5.2.2 Constraints from the bound upon the system energy 

In the case of a system with a finite number of configurations, Q1 + Q2 = ^(C/) — f (Co) is restricted 
to some interval [— |A£|,nax, +|A£|ijiax], and the definition (|5.28p entails the inequalities 

/pAiSi(t)+A2S2(t)\ 

As a consequence, 

ai^2(Ai, A2) = aa{Xa - Xb) with {a, b} = {I, 2}. (5.36) 
The relation (|5.36p can be rewritten as 

Q!i,2(Ai, A2) = a2(A2 — Ai) and ai(A) = q;2(— A). (5.37) 

The specific dependence of ai2(Ai,A2) upon A2 — Ai together with the generic formulae (j5.32p 
- (|5.33p imply the following relations between the infinite-time cumulants per unit time, 

lini -2l ^ (-l)P hm -2l (5.38) 

and 

lim -SiiSl = lim -S2_. (5.39) 

5.2.3 MDB and symmetry of the generating function for infinite-time cumulants per 
unit time 

The modified detailed balance entails the time-reversal symmetry (|4.8p for P {C f\Qi, Q2;t\Co) 
at finite time. Henceforth, according to its definition (j5.28p . the characteristic function can be 
rewritten as 

(gAiQi(t)-HA.S2(t)) =J2T. e^P'-^'^Q'+^P'-^-^Q' E E ^ (^o|2i> 22; t\Cf) P{Co,to = 0). (5.40) 

Qi S2 Cf Co 

When the Markov matrix is irreducible and the system has a finite number of configurations, the 
Perron- Frobenius theorem entails that there exists a single stationary state Pgt , and every config- 
uration C has a non-zero probability PgtiC). Henceforth, in the case where the initial distribution 
is the stationary one, the relation (j5.40p leads to the inequalities 



3(/3i-Ai)Qi + (/32-A2)Q2\ pi 



max 



< AT ^ < _st — ("5 41) 

pmax — (gAiQi+A2Q2)_,j — pmin ' V ■ / 

(We recall that (• • • )st denotes an average when the initial configurations are distributed according 
to the stationary measure Pst(C), the maximum and minimum values of which are P^^^ and P^™ 
respectively (see (|5.ip V These inequalities entail that the generating function for the infinite-time 
limits of the joint cumulants per unit time defined in (|5.3ip obeys the symmetry 

ai2(Ai, A2) = ai2(/3i - Ai, /32 - A2). (5.42) 

We notice that the symmetry (|5.42p can also be derived by considering the evolution of the 
Laplace transform of PiQi,Q2;t). With the notations of paper n|45], (e^i2i(*)+^222(t))_,^ ^ 
J2cj ^Co^^f\^^^^^ '^^^'^^\^o)Pst{Co) and q;i2(Ai,A2) coincides with the largest eigenvalue of the 
operator A(Ai, A2) which rules the evolution of V {e^\ e^'' ; t) according to dV/dt = AU. The MDB 
implies that the operator A(Ai,A2) obeys the symmetry A(Ai, A2) = A^(/3i — Ai,/32 — A2), where 
A"^ denotes the transposed matrix of A in the configuration basis. Then, by using the Perron- 
Frobenius theorem, one can prove that the largest eigenvalue of A(Ai,A2) satisfies the symmetry 
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relation (|5.42p (see the argument in [T] where an analogous symmetry is exhibited and then used 
for the derivation of the long-time fluctuation relation obeyed by the action functional defined in 
the comment after (|4.5l) - (|4.6p '). 

Moreover, since the system has a finite number of configurations, ai2(Ai, A2) = aa(Aa — Ab) by 
virtue of (j5.36p . and the symmetry property (j5.42p of cn2(Ai, A2) becomes 



aa{X) = aa(/3a -I3b-X) for {a, b} = {1, 2}. 



(5.43) 



We can apply the Gartner-Ellis theorem (see subsubsection [57TT51 and references there): if a2(A) 
exists and is differentiable for all A in R, then the large deviation function of the current J' = Q2/t 
exists and it can be calculated as the Legendre-Fenchel transform of 02 (A) , namely, with the signs 
chosen in the definitions used in the present paper, 



,{J) = min{a2(A) 
AeiR 



AJ}. 



(5.44) 



Then, since a2(A) obeys the symmetry (|5.43p . fQ2{J) obeys the fluctuation relation (|5.9p . We no- 
tice that we can similarly retrieve the fluctuation relation obeyed by /a„j.^,,s- Indeed, aA^xchSl'^) = 
limi_>+oo (1/0 111(6''''^™'''''^^*'') is equal to ai2(A/3i, A/32) with Q!i2(Ai, A2) defined in (|5.31l) . and, ac- 
cording to (|5.36p . aAexchSl-^) ~ (A(/3a — Pb))- Therefore the symmetry (|5.43p implies that 



,s(l - A) = aAexchs(A) 



(5.45) 



Under the above assumptions, the large deviation function of the exchange entropy cumulated 
current can be computed as the Legendre-Fenchel transform of aA^^chS' namely fA^^c^si'J) = 
minAeR{aAexchS'('^) ~ AJ^}, the latter symmetry allows to retrieve the fluctuation relation (|5.4p . 

Another consequence of the symmetry (|5.43p is that the Green-Kubo relation near equilibrium, 
which can be derived from the fluctuation relation (|5.9p for /q^ (J") as shown in section 15.1. 5[ can 
equivalently be directly derived from the symmetry (j5.43p of its Legendre-Fenchel transform 02 (A). 
This can be checked as follows. Relation (|5.43l) can be rewritten for a = 2 as 



a2(A;/3i,/32) = a2(--F- A;/3i,/32) 



(5.46) 



where T — Pi ~ is the thermodynamic force. The Green-Kubo relation involves the variation 
of the mean current J(/3i,/32) = ^)fl(A— o-/3 /3 ) ^^^^ respect to J-', so we need to compute mixed 
derivatives of a2 with respect to A and Notice that in the identity (|5.46p , the second and the 
third arguments are untouched. So if we make any invertible (A-independent) change of variables 
(/3i,/32) ^ i^^p) and set a2(A; /3i, /32) = a(A; J^, p), we have the following identity 



a{\; F, p) = a{—F — A; F, p) 



(5.47) 



where the third variable p is purely a spectator. Keeping it fixed, we get from the symmetry 
(ElTD: 



dFdX 

Taking this relation at A = 



= 0, we obtain 



a a 
'dX? 



(5.48) 



dFdX 



(A=0;jr=0) 



a a 
'dX? 



(5.49) 



From now on, the discussion parallels the one in section 15.1.51 We repeat the argument in a 
slightly different form to stress again the slight subtlety involved in variations with respect to 
in the context of thermal contact. The condition F = means /3i = /?2 = /3. So the right-hand 



side of (|5.49p is simply 



a' a 



(A=0;J^=0) 



0_Q2 

SA^ 



a a2 



(A=0;/3i=/32 = 



The left-hand side of (|5.49p can be 
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dealt with as follows. The change of variables (/3i,/32) ^ {^tP) is A- independent, so we can set 
A = once the derivative with respect to A is taken 



da 



(A=0;J^,p) 



da2 



(5.50) 



(A=0;/3i.fc) 



Of course, taking the partial derivative of J{f3i, (32) with respect to J- is ambiguous, as it depends 
on the choice of the variable p. However, ^|^_|^ has an intrinsic meaning, independent of 

the choice of the variable p. Indeed, by construction J(/3, /3) = (equilibrium) so — 



dJ 



/3i=/32=/3 



= 0. Moreover, from -F = /3i - ^2 we get 1 = |^ - 



This implies that f^l^^^g = ^ 



dJ 



is intrinsic and we can write (j5.49|) as 



9/3i 



=/32 

Nnw ILL - 9J dPi I dJ dl32 



dJ_ 



(5.51) 



(A=0;/3i=/32=/3) 



By using the relation (j5.33D for p = 2 between a2(A) and the infinite-time cumulant per unit time 
together with the fact that, when /3i = /?2, the stationary state is the equilibrium state, we get the 
Green-Kubo relation (|5.22p . We notice that the present derivation of the Green-Kubo relation is 
very similar to the argument developed by Lebowitz and Spohn in Ref.[I] in the case where there 
are several independent currents corresponding to several parameters which drive the system out 
of equilibrium. 



5.3 Far from equilibrium relations for infinite-time heat cumulants per 
unit time 

In the present subsection we settle generic results ; thus we replace the cumulated heat Q2 (t) by a 
generic cumulated quantity Xf. According to the Green-Kubo relation (|5.22p where, by virtue of 
(|5.23p . dJ/dJ-'\jr^Q can be replaced by lim_F^o J/J^^ in the vicinity of equilibrium the ratio J/T 
is equal to the equilibrium value of the second cumulant of the cumulated quantity Xt per unit 
time. On the contrary, when the system is far from equilibrium, dJ /d!F and J /!F are expected 
not to be equal to 1/2 times the second cumulant of the cumulated heat Xt per unit time in the 
stationary state. The property 

1^ ^ i linr m^^^ (5.52) 

arises from (|5.20p . while the discrepancy 

J 1 ^.^^ {X}),,-{Xt)l _ 

T 2 t^+oc t ^ ' 

is already mentioned in Ref.[lJ. This discrepancy implies that the generating function a{\) for the 
infinite-time cumulants of Xt per unit time is not a quadratic function of its argument A. namely 
the large deviation function of the current J7( = Xt/t is not a quadratic in the generic case, i.e. the 
probability distribution of Xt is not asymptotically Gaussian. Indeed, if the generating function 
a{\) were quadratic in A, i.e. ol{\) = a'^-'A -I- ^a^'^-'A^, then the symmetry (j5.47p would lead 
to a^^^ — ^a'-^-'J^. In the present section we derive an equation hierarchy for the infinite-time 
cumulants per unit time far from equilibrium, which in particular gives how J is related to all 
even cumulants per unit time in the infinite-time limit. 

5.3.1 Generalized Green-Kubo relations for cumulants per unit time far from equi- 
librium 

The starting point is (|5.46p which we rewrite for convenience in the generic form 

a(A; J") = a(-J'- A; J"), (5.54) 
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where T is the thermodynamical force (an inverse temperature difference in (j5.43p ). The viewpoint 
is that the coefficients in the expansion of a(\\ F) in powers of A at A = are related to the heat 
cumulants per unit time in the infinite-time hmit, 

= Um -K^\t), (5.55) 

t->+oo t 

and these cumulants per unit time are experimentally measurable. This is true at least for the 
first few ones. 

There are a number of ways to rewrite the symmetry (j5.54p . Formally it is equivalent to 

e"-^^a(A; F) = a(-A; T). (5.56) 

Expanding both sides in powers of A and comparing we get that the cumulants per unit time, 
given by a{\\F) = J2p=^ ^'^^'^'^'(•7^) obey the equation hierarchy 

kM(^) = ^(-1)p+«^k[p+?1(^) forp = 0,l,--- . (5.57) 
9=0 ^■ 

In (|5.57p k[°1(J") = 0, because a{X;T) is a cumulant generating function. The relation (|5.57p can 
be explicitly checked in the case of the solvable model studied in paper II [5S] in a kinetic regime 
where the probability distribution of the heat received from the slow thermostat is that of an 
asymmetric random walk. 

A further expansion of the equations (j5.57p in powers of with k'^I (T) = ■^T'^k^^''^^ , 

yields another hierarchy of relations, each of which involves only a finite number of derivatives of 
cumulants per unit time with respect to J-, 

^[p-.g] ^ ^(-1)P+'- f'l\T^lP+r:'}-r] for g ^ 0, 1, • • • , (5.58) 

where («) ee q\/[rl{q - r)!], and k^'P'°^ = k[p1(J' = 0) while for q > 1 

gq-i^lp] 



-f.lp;ci] 



(5.59) 



From the cumulant generating function interpretation, in (j5.58p it is to be understood that k^'^^'?! = 
for q = 0, 1, • • • , since k^°^ [F) = 0. 

The relation hierarchies (|5.57p and (|5.58p are pure consequences of the symmetry (|5.54p without 
using any further properties of a. Anyway, they express nothing but a parity relation around 
A — J- 12, which fixes half of the coefficients in terms of the other ones, so the equations are not 
expected to be all independent. Indeed, this is particularly transparent at small order in T : for 
q = (j5.58p yields nothing if p is even, but if p is odd it leads to — Q. As for the hierarchy 

(|5.57p obtained before expansions in powers of J^, the dependence between its equations can be 
exhibited as follows. By specifying the equations to p = 2n ot p — 2n + 1, the infinite system of 
equations (I5.57P can be rewritten as the equivalent hierarchy. 



7^P»+ll(^) = ^(-l).+ l_^7^[2n+.+ l](^) 



+ 00 

-[2„+l](_^) = i^(-l)9+l:^^K[2"+9+l](^) 



forn = 0,l,---, (5.60) 



where the right-hand sides of both relations involves both odd and even cumulants per unit time. 
The difference between the two expressions for (jt^ gives the infinite sum rule 
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It is not difficult to obtain independent relations which express odd cumulants in terms of even 
cumulants. We rewrite (|5.56p as 

1 + e--^*) a(A; F) = a(A; F) + a{-X; J"), (5.61) 

a{X;T) = ^-^[a(A;J-) + a(-A;-F)]. (5.62) 

1 + e^^Tn: 

Note that [1 + e^^] ^ + [1 + e^] ^ = 1 so that [1 + e^^] ^ — ^ is an odd function of x. Hence the 
Taylor expansion in x can be written 



\ fc=0 / 



,1 + -- — + — -^^ + --- ), (5.63) 
2 V 2 24 240 40320 ' ^ ^ 



where the coefficients dk are related to the classical Bernoulli numbers by 



From the identity 



2 

one infers the recursion relation 



l = i(l + e--) l + (5.65) 



k=0 



Finally, in the expansion of (|5.62p the even terms cancel out and one is left with 

-[2„+i] ^ J2 dfe^2fe+i^[2(«+fc+i)] for n = 0, 1, • • • , (5.67) 

fc=0 

which expresses systematically odd cumulants per unit time in terms of even cumulants per unit 
time. For instance, in the case n = 0, the ratio of the out-of-equilibrium current J{J-) = k'^^ and 
the thermodynamic force J- is determined by all even cumulants as 

'^-T.V'\};fJ—r^. (5.68) 

The latter relation may be viewed as a a far-from-equilibrium generalization of the Green-Kubo 
relation 

5.3.2 Relations between non-linear responses of cumulants per unit time near equi- 
librium 

It is to be noted that the two equivalent hierarchies (j5.57l) and (I5.67P for relations between cumu- 
lants make sense only if the cumulants satisfy some growth condition, the second one being most 
stringent (note that dq ~ 4(— l)'7r~^''~^). Such problem do not arise in the double expansions in 
both A and T. Expanding (|5.67p in powers of J-' yields for n = 0, 1, • • • 



^[2»+i;0] ^ (5.69) 

L(9-1)/2J , , 

-[2n+l;g] _ "fcg- -f2(n+fc+l);<,-(2fc+l)] „ _ 1 O . 
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where [yj denotes the lower integer part of y. It is easy to check, at least for small values of 
q, that the contents of (j5.58l) and (j5.69D are the same. The physical consequences of the latter 
relations are more readily inferred by explicitly rewriting the relations in the case where q is odd 
or even and in terms of either equilibrium cumulants per unit time or the partial derivatives of 
out-of-equilibrium cumulants per unit time with respect to the thermodynamic force J-. Then the 
formulae (j5.69p read 

(5.70a) 
(5.70b) 



5^[2n+l] 



Q2m-^[2n+l] 



(2m)! 92 



-l^[2(n+m+l-r)] 



(2r- 1)! 



(5.70c) 



T=0 



g2m+lj^[2n+l] 



Qjr2m+1 



T=0 



m>l 



dm(2m + 1)! K, 



[2(ri+m+l)] 
eq 



dm. — '. 



(2m + 1)! a2r^[2("+m+l- 



r)] 



(2r) 



J^=0 

(5.70d) 

where the values of the rf^'s are given in (|5.64p . 

The equilibrium statements (|5.70al) are exemplified in the case of thermal contact as follows. 
The first cumulant per unit time k^^I for the cumulated heat Q2 coincides with the average of 
the instantaneous current of j2; 02)sf^^^'' = J: and from the first equation (I5.70ap for n = we 
retrieve that at equilibrium the mean instantaneous current vanishes, Jeq = 0. More generally, 
(I5.70ap states that at equilibrium all odd cumulants per unit time of the heat amounts received 
from one of the two heat baths vanish. 

The relations (j5.70bp are generalized fluctuation-dissipation relations in the vicinity of equi- 
librium, which are called "generalized" in the sense that they express the linear response of any 
cumulant per unit time. Indeed, in the case of thermal contact, from (I5.70bp for n = one retrieves 
the fluctuation-dissipation relation (also called Green-Kubo relation) in the form (|5.22p where Xt 
is the heat amount received from one of the two heat baths. More generally, from (|5.70bp for any 
value of n one gets the generalized fluctuation-dissipation relations 



lim — 

A)^(/3,/J) Pi 



1 



/32 



lim 



t 



1 ,. K, 

= — lim 
2 



2n+2] 
cq 



t 



(5.71) 



Analogous relations have been derived in the case of several independent out-of-equilibrium steady 
currents by Andrieux and Gaspard [HI]- We notice that for n = 1, (|5.7ip means that, in the limit of 
vanishing /3i — /32 , the ratio between the out-of-equilibrium third centered moment (non- normalized 
skewness) ((Q2 — {Q2))^) per unit time and the thermodynamic force (3i — /?2 tends to half the 
equilibrium fourth cumulant (kurtosis multiplied by the square of the variance) ((Q2 — (Q2)cq)^)eq— 
3((Q2~ (Q2)eq)^)eq per unit time. These cumulants are expected to be experimentally measurable. 

The relations (I5.70cp and (j5.70dp deal with higher-order response coefficients. The relation 
(|g7Dc| in the case m = 1 reads d^K^^''+^^ /dT^\jr=o = &k^^'^+'^^ /dT\jr^o. For instance, for n = 
and n = 1 it leads respectively to 



and 



Q2j.m 



The relation (I5.70dp in the case m = 1 gives d^n'-' 
For instance, in the case n = it reads 



|927.[2n+2]/aj-2 



J^=0 ■ 



3 927.[2] 

2 



(5.72) 



(5.73) 



_[2n+4] 



(5.74) 



F=0 
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6 Extension of previous results to stationary states with sev- 
eral independent currents 

In this section we consider the generic case where a system exchanges several kinds of micro- 
scopic conserved quantities with several reservoirs and each reservoir exchanges only one kind of 
these extensive quantities. Moreover, if several reservoirs are involved in a given transition, they 
should be reservoirs for distinct, independent, physical quantities (this condition is required in 
the derivation of the MDB.) In the following s is the index of a species of conserved exchanged 
quantity (energy, volume, or number of particles . . . ), while a is the index of a reservoir. In this 
situation the microscopic exchange entropy variation i5exch'S'(C' <— C) when the system jumps from 
configuration C to configuration C reads 

<5cxch^(C' ^ C) = Y,FaSxa{C' ^ C). (6.1) 

a 

The sum runs over all reservoir indices. Fa is the value of the intensive parameter which charac- 
terizes the reservoir with index a which is at thermodynamic equilibrium, and Sxa{C' ^ C) is the 
extensive quantity received by the system from the reservoir a when the system jumps from C to 
C, with the same convention as in definition (|2.20l) . 



6.1 Consequences of MDB at finite time 

In the form (jl.3p where it involves the microscopic exchange entropy variation ScxchSiC' ^ C) 
associated with a jump from configuration C to configuration C the MDB entails the symmetry 
(j4.6p between the probabilities for time-reversed histories. At a more mesoscopic level, let us 
compare the probability of evolutions from the configuration Co to the configuration C/, in the 
course of which the system receives a cumulated quantity Xa ~ "^2 from each reservoir a, and 
the probability of the reversed evolutions, namely evolutions from Cf to Co where the cumulated 
quantities are —XaS. The symmetry (j4.8p written in the case of thermal contact becomes in the 
generic case 

P{Cf\{Xa};t\Co) ^ -A<,,,hS({X„}) (p^ o\ 

p{Co\{-xMCf) ' ^ ■ > 

with 

Acxch5({Xj) = J2 FaXa. (6.3) 

a 

At the macroscopic level, namely when only the exchanges of extensive quantities with the 
reservoirs are measured, there appears a symmetry for transient regimes where the system initially 
prepared in some equilibrium state is suddenly put into contact with reservoirs with thermody- 
namic parameters FaS that drive the system into a non-equilibrium state. The symmetry involves 
the excess exchange entropy variation AI^J^S", defined as the difference between the exchange en- 
tropy variation under the non-equilibrium external constraints -Fa's , namely "Ylia^o-Xa, and the 
corresponding variation under the equilibrium conditions where for all reservoirs which exchange 

is) 

the same species s the thermodynamic parameter has the same value Fq , 

AS^S({^a};{Fo(^^})^A_h5({Xj)-^F(^' ^ Xa, (6.4) 

where R{s) is the set of reservoirs which exchange the microscopic quantity of species s. When 
the system is prepared in the equilibrium state with probability distribution Peq({Fo*'}) and 
put into contact with reservoirs with thermodynamic parameters Fa's at the initial time of the 
measurements of exchanged quantities, the excess exchange entropy variation AJJ^^IS* obeys the 
symmetry relation at any finite time, or "detailed fluctuation relation", 

Pp f/i?( = )\) (Acxch'5') .oxcso 

-^P,,({Fo(='}) ^ ^exch-^J 
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The latter relation itself entails the identity, or "integral fluctuation relation", 



<^^"^^^')p.({.^=>}) - 1- (6.6) 
6.2 Consequences of MDB in the infinite-time limit 

A generalization of the argument in subsection 15.1.11 shows that the symmetry relation (|6.2p 
enforced by the MDB at finite time leads to the existence of lower and upper bounds for the 
ratio between the finite-time probability to measure cumulated quantities with values {Xa} and 
the corresponding probability to measure the opposite values {—Xa}, when the system is in its 
stationary state. The finite-time inequalities (j5.ip become in the generic case 

< Pst {{Xa};t) ^ 

" ^'st ({-Xj;Oe-^exc.s({x„}) - pmin • y ■ ) 

As a consequence, the dimensionless exchange entropy variation Acxch'S' given by (j6.3p obeys the 
fluctuation relation (|1.8p . 

As for the characteristic function of the extensive exchanged quantities X^s in the stationary 
state, namely (e^^ ^""^"(*))st, the symmetry (|6.2p - (|6.3p arising from the MDB entails that the 
characteristic function obeys an inequality similar to (|5.4ip . As a consequence, the generating 
function of the infinite-time limit of the joint cumulants per unit time obeys a symmetry which is 
a generalization of (j5.42p 

«{X„}({Aa}) - 0^{XM{-Fa - Xa}). (6.8) 

However, because of the conservation laws for the microscopic quantities exchanged with the 
reservoirs, the joint cumulants of the X^'s per unit time are not independent in the infinite-time 
limit. For instance, if R{s) denotes the set of reservoirs which exchange the microscopic quantity 
of species s, because of the conservation law for every species of exchanged quantity, X]ae-R(s) 
is equal to the difference between the values of some observable of the system in the final and 
initial configurations of the evolution. When the system has a finite number of configurations, 
J2a£R{s) -^a. is boundcd and the mean currents J^'s defined as Ja = limt_i.+oo(Xa(t))/< are related 
by 

J2 ^- = 0- (6-9) 

ae-R(s) 

Other relations may arise from other microscopic conservation rules determined by the specific 
forms of the transitions rates. We consider the generic (but not universal) case where there exists 
a set of cumulated quantities Ky's, which are linear combinations of the Xa's, but less numerous 
than the Xa's, and a set of parameters T^'s such that J^a^^Xa — '^■y-^-y^-y bounded by a 
constant independent of time. With these assumptions the expression for the exchange entropy 
flow in the stationary state takes the generic form 



^cxch 



s 



dt 



= -J2PaJa = -J2^yJl^ (6.10) 



where the mean currents J*'s, defined as J* = \imt-^^ao{y-y{t)) /t, are less numerous than the J^'s. 
Moreover, these assumptions entail that the symmetry (I6.2l) - (|6.3p leads to the inequality 

m< ^y/ITO;^l^o) <M, (6.11) 

- P{Co\{-Y,};t\Cf)e-^-.^-'^-' " 

where m and M are constants. As a consequence, the characteristic function for the l^'s obeys 
an inequality similar to (|5.4ip and subsequently the generating function of the infinite-time limits 
of their joint cumulants per unit time obeys a symmetry which is a generalization of (j5.42p . 

«{y.}({A7};{-^7}) = «{y.}({--^7-A7};{-^7})- (6-i2) 
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In the framework of graph theory (see subsection 13.3. 5p Andrieux and Gaspard [33] have shown 
an analogous symmetry for dimensionless cumulated quantities Ky's, where each is defined as 
the sum of the cumulated currents through the chords of all cycles in the graph which have the 
same affinity A-^ and where the JT^'s are replaced by the cycle affinities A^'s. A similar result 
is also obtained in Ref . |72| . The latter symmetries are valid even when several reservoirs which 
exchange the same species can make the system jump from one configuration to another one, as 
it is the case for coupled chemical reactions. 



6.3 Generalized Green-Kubo relations for several independent currents 

In standard fluctuation-dissipation Green-Kubo formulae, valid near equilibrium, the relevant out- 
of-equilibrium quantity which is related to equilibrium fluctuations is the Onsager coefficient La-y 
introduced in phenomenological thermodynamics of irreversible phenomena as the derivative of a 
" thermodynamic flux" J* with respect to a "thermodynamic force" J%y, which are associated one 
to another from the expression of the entropy production rate d{-a_tS / dt\st — —dc-x_c\iS / dt\st given 
by (j6.10p when the J*'s are independent currents between reservoirs. The Onsager coefficient is 
defined as 



(6.13) 



and the generic statement of Green-Kubo relations reads 

2 t^+oo t ■ 

Therefore in the case where the non equilibrium stationary state involves several independent 
stationary currents, the symmetry (Ya{t)Y^{t)) = (y-y{t)Ya{t))eq allows to retrieve the phe- 
nomenological Onsager symmetry for the off-diagonal Onsager coefficients, namely La-y — L^a- 

Far from equilibrium generalized Green-Kubo relations can be derived from the symmetry 
(I6.12p of the generating function for the infinite-time cumulants per unit time. The generalization 
of the combinatorics considerations of subsection 15.31 is straightforward. Suppose the independent 
currents are indexed by 7 € F so that instead of a single parameter A one deals with a collection 
A = (A^)-ygr- In the same way, one has a collection ^ = (J^y)-yg^• However, one considers a single 
function a and we rewrite (|6.12p as 

a(A;Z) = a(-Z- A;Z). (6.15) 



One could redo all the derivations performed in subsection 15.31 We content to express all odd 
cumulants in terms of the even ones. Write the expansion in powers of the A-^'s in a compact way 

as 

a(A;Z)s V^A^i«;W(Z), (6.16) 



p>o- 



where the summation is over F-tuples p = (p^)^gr of non-negative integers, p\ = Yi-ferPf^- ^^'^ 
A^= Note that 



al£la(A;Z) 



9A£ 



(6.17) 



A=0 



where \p\ = ^^^yPj = n7er^'^7"^- expand the symmetry relation written in the 

form 

a(A;Z) = ^— r [a(A;Z) + a(-A;Z)] , (6.18) 



TT (J 



where 

- 76r ^ 
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From 

+ e - \ k=0 p>0,|p|=2fe+l - — / 

one infers that, for n such that \n\ is odd 

75[^(Z) = E'^'= E (2fe + ^)' j;£7;[i^+£](j;). (6.21) 

fc=0 P>0,|p|=2fc+1 -' 

Again, these relations can be expanded in powers of The corresponding coefficients of the 
powers in are the non-linear response coefficients. The relations between the latter non-linear 
response coefficients are derived by another method in [ST]. For instance relations analogous to 
(j5.70d[) relate non-linear response coefficients of the cumulants per unit time caused by a variation 
in thermodynamic forces to some equilibrium cumulant per unit time, which is symmetric 
under permutations of the associated cumulated quantities. The latter symmetry is at the origin 
of Onsager reciprocity relations in the case of Laj [I] (see definition (I6.13P ) and of generalized 
symmetry relations for higher-order response coefficients of cumulants per unit time as noted by 
Andrieux and Gaspard |73[ I51| . 

A Property of the coarse grained dynamics 

In the present subsection we derive the property (|2.5p valid over a period of the ergodic determin- 
istic microscopic dynamics T when T respects both the conservation of i?doc and the interaction 
pattern specified at the end of section [2.1.2l 

Though we believe that a general development could be pursued, we prefer to concentrate on 
a specific model at this point. The system is made of two large parts and a small one, which 
is reduced to two Ising spins ai,a2 — ±1, each one directly in contact with one of the large 
parts. So a configuration C can be written as C = (Ci, cri, 172, (^2)- We assume that, when the 
small part is isolated, its energy £{(Ti,a2) does not describe independent spins. Moreover, the 
microscopic dynamics T conserves the energy Edec{C) where the interactions between parts is 
neglected, namely i?dec(C) = -Bi(Ci) -I- £2(02) + f ((Ji,CT2). The remnant of interactions between 
the parts is embodied in the following restrictions. For any C, T(C) is obtained from C by one of 
the following operations : 

- (I) a flip of spin ci together with a change in Ci and a possible change in C2 such that 
Ei{Ci) -f £'(cri, (T2) and £2(02) both remain constant (i.e. the energy needed to flip the spin ai 
entirely comes from or goes to the large part 1). 

- (II) a flip of spin (T2 together with a change in C2 and a possible change in Ci such that 
Ei{Ci) and £2(02) + £{<^i,0'2) both remain constant (i.e. the energy needed to flip the spin (T2 
entirely comes from or goes to large part 2). 

- (Ill) a change in Ci and/or C2 but no flip of cri or (T2, such that £'i(Ci) and £2(02) both 
remain constant, as well as £{ai,a2)- 

In order to build an effective mesoscopic dynamics we just keep track of (E'l, cri, CT2, £^2) as a 
function of time. Therefore during the time evolution of a given configuration C of the whole 
system we concentrate only on time steps at which a change of type (I) or (II) occurs, namely 
when either spin ui or spin (T2 is flipped with known corresponding variations in Ei and £2- We 
do not follow precisely the changes (III) that modifies the configurations of large parts without 
changing the energy of any part (either E2 or £{ai,a2))- The possible changes are 

iEuai,a2,E2)'^ {E[,-ai,a2,E2) with E[ ^ Ei + £{ai,a2) - £{-<7i,a2) type (I) (A.l) 
and 

{Ei,ai,a2,E2)^ iEi,ai,-a2,E'2) with E'2 ^ E2+£iai,a2)-£{ai,-a2) type (II) (A.2) 
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Starting from some initial configuration (iJj', ctj', ctj , i?2)i some set of possible (£'1,(71,172,^2) 
will be visited during the time evolution over a period of T , and it is useful to view this set 
as the vertices of a graph, whose edges connect two vertices if one can jump from one to the 
other by a single change of type (I) or (II). This graph can be chosen to be unoriented because a 
transformation of type (I) or of type (II) is its own inverse. Then a trajectory over a period of the 
underlying deterministic dynamics corresponds to a closed walk on this graph, which summarizes 
the coarse graining due to the macroscopic description of the large parts. During a period of the 
microscopic dynamics the closed walk on the graph goes through each edge a number of times. 

The main observation is that, since energy £((Ti,(T2) does not describe independent spins, the 
graph has the topology of a segment. Indeed, the graph is connected by construction, but each 
vertex has at most two neighbors. So the graph is either a segment or a circle. Let us suppose that 
it is a circle. Then one can go from (£", (T°, C2, -E'2) to itself by visiting each edge exactly once, 
i.e. by an alternation of moves of type (I) and (II). After two steps both spins in the small part 
are flipped, so the total length of the circle is a multiple of 4. But after 4 steps a definite amount 
of energy has been transferred between the two large parts, namely the energy in the first large 
part has changed by £{(Ji, CT2) — ^■{—o'l , (^2) + £{—<^i^ —<^2) — £i<^i, ~o'2) and a trivial computation 
shows that this cannot vanish unless £(cri, (T2) describes two independent spins, a possibility which 
has been discarded. This excludes the circle topology. 

Since the graph is a segment, any closed walk on the graph traverses a given edge the same 
number of times in one direction and in the other one. As a consequence, during a period of 
the microscopic dynamics T, the motion induced by T on the graph is such that the number of 
transitions of type (I) (i?i, CTi, (T2, -B2) [E'l, —cri, (72, E2) is equal to the number of their inverse 
transitions — cti, (72, E2) — > (£1, ci, (T2, E2), where the relation between E[ and Ei is given in 
(|A.1[1 . The same considerations apply for transition rates associated with the flipping of (T2 . The 
result is summarized in (|2.5p . 

For a more general discrete system, the mesoscopic time evolution can again be characterized 
by the time steps where the small system variables are flipped while the changes in the large parts 
are only macroscopically described by their net energies. An analogous graph can be constructed 
but it can be much harder to analyze its topology, which is the crucial knowledge needed to 
exploit the consequences of ergodicity. These consequences are most stringent for a tree. There is 
no reason a priori why the graph should be a tree, but what this implies, namely that ratios for 
the transition rates involving the small part and a large part are given by ratios of energy level 
degeneracies in the large part, is physically quite appealing. 

B The Markovian approximation 

This appendix is a short digression on mathematics. The aim is to briefly recall a trick allowing 
to replace a fixed sequence of symbols by a random one with "analogous" statistical properties, 
and then to combine this trick with a coarse graining procedure. 

B.l Discrete time stochastic approximation 

Suppose w = X1X2 • • ■ xn is a given finite sequence of elements of a finite set S. It is convenient to 
assume that this sequence is periodic, i.e. that xn+i = xi. For x £ S, set = #{« £ [1, N],Xi — 
x}, i.e. Nx is the number of occurences of x in the sequence w. There is no loss of generality in 
assuming that ^ for every x G S, should it lead to consider a smaller 5*. For x,x' G S set 
Nxx' = £ N],Xi — X, Xi+i — x'}, i.e. N^x' is the number of occurences of the pattern xx' is 
the sequence w. Notice that we accept that x ^ x' in this definition. Of course, we could look at 
the occurence of more general patterns. By definition, we have '^x'es ^xx' = Saj'sS ^^'^ ~ 
The result we want to recall is the following. 

There is a single time-homogeneous irreducible Markov matrix on S which fulfills the following 
two requirements. First, in the stationary state of the corresponding discrete-time stochastic 
evolution, the probability Pstix',i + l]x,i) that a sample x takes the value x at time i and the 
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value x' at time z + 1 is equal to the frequency of the pattern xx' in the sequence w, namely 

P,t(a;',z + l;x,i) = (B.l) 

Second, the stationary probability that the random variable x takes the value x at any time i is 
equal to the frequency of x in the sequence it;, namely 

PA^) = ^- (B.2) 

In fact (jB.2p is a consequence of (jB-lf) . because of the relations Pst{x) — J2x'£S Pst{x' , i + l;x,i) 
and J2x'es ^x^' ~ words, there is a unique time-homogeneous irreducible Markov 

chain whose stationary statistics for patterns of length 1 or 2 is the same as the corresponding 
statistics in w. 

The proof is elementary. By the Markov property, the Markov matrix element from x to x' , 
denoteco by T{x' ^ x), must satisfy the relation Pst(x', i + l;x,i) = T{x' ^ x)Pst{x)^ so the only 
candidate is 

T{x'^x)^^. (B.3) 

Conversely, the corresponding matrix is obviously a Markov matrix (X^x'gS -^(■^' ''^ x) — \ since 
X^x'es ^xx' — Nx), and it is irreducible, because as w contains all elements of S, transitions within 
w allow to go from every element of S to every other. Checking that for this Markov matrix the 
stationary measure, namely the solution of '^^^gT{x' x)Pst{x) — Pst{x'), is given by (|B.2p 
boils down to the identity J^xes ^^x' = Nx' recalled above, and then the two-point property (jB.ll) 
follows. This finishes the proof. 

In the case very specific case where xi,X2^ ■ ■ ■ ,xn are all distinct, i.e. \S\, the cardinal of S, 
is equal to N, Nx — 1 for all a; € S" and Nxx' = Sx',xi-^i where i is such that Xi = x. Then the 
only randomness lies in the choice of the initial distribution, and each trajectory of the Markov 
chain reproduces w up to a translation of all indices. If N is large and \S\ is ~ N, slightly weaker 
but analogous conclusions survive. A more interesting case is when \S\ <C N by many orders of 
magnitude. 

This trick has been used for instance to write a random text "the Shakespeare way" by com- 
puting the statistics of sequences of two words in one of his books. 

By definition, the Markovian approximation reproduces the statistics of the original sequence 
only for length 1 and length 2 patterns. It is a delicate issue to decide whether or not is also does 
a reasonnable job for other patterns. For instance, the random Shakespeare book certainly looks 
queer. Various physical but heuristic arguments suggest that for the kind of sequences w relevant 
for this work, the Markov approximation is quite good, but we shall not embark on that. 



B.2 Continuous time approximation 

By the very same argument, there is a single time-homogeneous irreducible Markov transition 
matrix on S and a single time scale r such that, in the stationary state of the corresponding 
continuous-time stochastic evolution, the expected number of transitions from x to x' ^ x per 
unit time is Nxx' /t. The formula one finds for the transition rate is 

N I 

Wix' ^ x) = for X ^ x' . (B.4) 

tNx 

The continuous-time approximation becomes more natural in the case when \S\^ the cardinal 
of S*, is such that \S\ ^ iV, while for all x Nxx ~ Nx and for all x ^ x' Nxx' ^ Nx, which means 
that transitions are rare and most of the time x follows x in the sequence w. Again, one can argue 
that for the kind of sequences w relevant for this work, this is guaranteed by physics. Then taking 

*In the whole paper we use the convention that the evolution (here given by the transition matrix T) is written 
from the right to the left as in quantum mechanics. 
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T (in some macroscopic time unit) of the order of the largest of the N^x'/N^, x ^ x' , one ends 
with a Markov transition matrix with elements of order unity (in some macroscopic inverse time 
unit), and U = ri can be taken as the physical macroscopic time. 

To summarize, in this work, we shall systematically associate to certain sequences w a continuous- 
time Markov process and exploit properties of w to constraint the structure of W{x' x). 

A natural application of the above ideas is to the case where the sequence w arises from some 
coarse graining procedure. One starts from a sequence uj = ^1^2 ■ • • Ctv where ^1 , ^2, ■ ' ' > belong 
to a set S that is so large that lu cannot be stored, and that only some of its features can be 
kept. Say we partition S = Uxi^s'^x where S is of reasonable size. Then all we keep of uj is 
w = X1X2 • ■ -xn where, for i e [1;-^]) xt is substituted for S,i when e "^xi- In the applications 
we have in mind, S is an A'^-element set, i.e. all terms in the sequence w are distinct. In that 
case, even if uj was constructed in a perfectly deterministic way, by saying who follows who in the 
sequence, such a description is unavailable on w, and w may well look quite random, so the Markov 
chain approximation is worth a try. In fact. jS"! ^ iV and we shall take as a physical input that 
transitions are rare, so that the (continuous-time) Markov process is an excellent approximation 
to the (discrete time) Markov chain. 

C Microcanonical detailed balance and time reversal invari- 
ance in Hamiltonian dynamics 

In view of comparison with the approach developed for discrete variables, we rederive the micro- 
canonical detailed balance (j2.9p in the case where a microscopic configuration of the full system is 
described by continuous variables and the system has a microscopic deterministic dynamics whose 
Hamiltonian H , independent of time, is an even function of momenta. (Similar arguments can be 
found in derivations which rely on different assumptions in Refs. |741 148j .l 

Further hypotheses are the following. The coordinate of the system in phase space is denoted 
by ^, and x[^] is a set of mesoscopic variables defined from the microscopic coordinate ^ and which 
are even functions of momenta. The initial position of the system in phase space is not known, 
but we assume that it is uniformly randomly distributed in the energy shell E = H[£], where E 
is the value of the energy of the full system at some initial time to. In other words, the initial 
probability distribution of ^, Po{£,), is such that 

where rf^ is the Liouville measure in phase space. 

The dynamics is invariant by time translation and we simply denote by /t(^) the position of 
the system in phase space at time to + t knowing that it is at position ^ at time to- (For the 
same reason, in the following the initial time to is set equal to 0). Then the invariance of the 
Hamiltonian under the operation R which changes every momentum into its opposite implies that 

[RftRfm^i, (C.2) 

namely the microscopic dynamics is invariant under time reversal. Since the Hamiltonian is a 
constant of motion and the equations of motion conserve the infinitesimal volume in phase space, 
the probability (i^-Po(C) defined in (IC.1|) is also conserved under the microscopic evolution, 

d{fmPoUtm)=d^Po{0- (C.3) 

When the initial position of the system in phase space is distributed according to Pq, the 
probability that at time t the set of mesoscopic variable x takes the value Xi , which is defined as 

Ppoi2Li,t) ^ f d^Po{OH2l[ft{0]-2Ll), (C.4) 
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is in fact independent of time (i.e. conserved by the microscopic dynamics). Indeed, let us consider 
the change of variable ^' — ft{£,)- The conservation of PQ{(.)d(^ by the dynamics (jC.3p implies that 
d^Po{^) = d^' Po(C')- Then the integral in (fOU reads / d^' Po{C) S - x^). It is independent 
of time and, by virtue of the definition (IC-H) of the distribution Po{^) in phase space, it is in fact 
equal to the microcanonical distribution -Pmc(aii), 

On the other hand, the probability that, again when the initial position of the system in phase 
space is distributed according to P(){£,), the set of mesoscopic variables x takes the value Xi at 
time t — and the value X2 at time t reads 

Ppo(£2,i;£i,0) = J dCPoiOH3L[ftm-3l2)Hm-^i)- (C.6) 

Since the variables denoted by x{^) are even functions of the momenta, we can write x[ft{^)] — 
x[[Rft]{£,)]- Let us consider the change of variable ^' = [Rft]{£,). The invariance under time 
reversal of the microscopic dynamics ([02]) entails that $ = [Rft]{^'), so that = x[[Rft]{C)] = 
x[ft{0]- By virtue of its definition (jC.ip d^ Po[£,] is invariant under the operation R, so d^' Po{£,') = 
d{ft[£,]) PoiftiS,]) and the conservation of Po{^)d^ by the dynamics (jC.3|) implies that d£,' Po{£,') = 
rf^Po(0. Eventually the integral in (|C?6l) reads / dC PoiOH^iC] - £2) ^ (£[/t(C')] and we 

get 

Ppo{2l2,t;ili,0) ^ Pp,ix^,t;x2,0). (C.7) 

The joint probability is invariant by exchanging the times at which Xi and X2 occur. 

We now assume that the evolution of the probability of the mesoscopic variable x, which is 
invariant by time translation, can be described by an homogeneous Markovian stochastic process 



whose stationary distribution Pgt (x) is the time- independent probability Pp^ (x) defined in (|C.4|I 
and whose transition rates, denoted by W{x' x)^ are determined by the identification 

Ppo dt; x, 0) = W{x' ^ x)Pp„ (x) X dt, (C.8) 

where the time-displaced joint probability for x^ and x and the probability of the single variable 
X are calculated with the same initial distribution Pq for the microscopic variables ^'s. Then, the 
time reversal symmetry property (|C.7p for the joint probability and the fact that Ppg {x) coincides 
with the microcanonical distribution (x) by virtue of (|C.5P lead to the microcanonical detailed 
balance 

Wix' ^ x)Pn,cix) = Wix ^ x')Pn,cix). (C.9) 



D Definitions for statistics over histories 

Consider a history where the system starts in configuration Cq at time to ~ and ends in con- 
figuration C/ at time t after going through successive configurations Cq, Ci,. . . , Cn = Cf. The 
N instantaneous jumps from one configuration to another occur at N successive intermediate 
times Ti which are continuous stochastic variables : the system jumps from Ci-i to Ci at time Ti 
{i = 1,. . . ,N) in the time interval [ti,ti + dti[, with to < ti < t2 < ■ ■ ■ < < t. The probability 
measure for such a history is related to the probability density Ilcf,Co [Hist] by 

dPcf^Co [Hist] =dti... dtNllcf^Co [Htst] (D.l) 
where, for a time-translational invariant process, 

Ilcj,Co[Hist] = e''^'''"'>^'^^"\CNMCN-i)e-^'"-'"-''>^'^^"-''> (D.2) 

X . . . e-(*^-*i)^('^^)(Ci|W|Co)e-(*i-*«)'^('^°) 
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and A(C) is the total exit rate from configuration C, 

A(C)^-5^(C'|W|C). (D.3) 

C'jiC 

The average of a functional FlT-List] over the histories which start in configuration Co and end 
in configuration C/ is computed as 

{F)cf,Co = j dPcf,Co [Hisi\F[Hist] (D.4) 

with 

/ +00 „ 

dPc, ,Co [Hist] = J2J2---T. ^^c„Co [nist] . (D.5) 

Then the average of a functional when the initial distribution of configurations is Pq reads 

{F)p, =J2Y1 ^o(^o) / '^^c.A [Hist] F[Hist]. (D.6) 

Cf Co 

E Some remarks on large deviations 

Suppose that Xt is some time-dependent random quantity. Typically, what we have in mind is a 
variation of some physical quantity (heat, entropy) exchanged between a reservoir and a system 
during the interval [0,t]. So Xi is expected to scale like t at large times, with an average {Xt)/t 
going to a constant J as t goes to -|-cxd. 

There are a number of definitions to quantify the probability that Xt/t differs significantly 
from J at some large time. It often happens that this probability is exponentially small, and a 
general mathematical theory, large deviation theory, has emerged to describe this situation. 

E.l The mathematical definition of Icirge deviations 

The general definition is a bit abstract, dealing with general probability measures depending on t. 

Wc restrict here to the situation when the probability measure is the distribution of a real random 
variable Xt- If j_ < j+ are two real numbers, observe that P j+Q < P e 

so that 

- ilnP (^^ e < -ilnP (^^ e]i_,j+[) . (E.1) 

For large t, the two members may not converge but at least 

lim inf - 1 In P f ^ e [i_ , j+]) < lim sup - ^ In P f ^ e] j_ , j+ [) , (E.2) 

t-^+co t \ t ) t->+oo t \ t J 

where both sides of the inequality belong to [0, +00]. A nontrivial lower bound for the left-hand 
side or upper bound for the right-hand side gives information on the exponential rate of decrease of 
P € j+]) and P the best situation being when the two exist and coincide. 

One says that the random variable X( obeys a large deviation principle (LDP) if there is a 
lower semi-continuous function Rx, called the rate function, or large deviation function, such that 

inf Rx{3) < liminf-ilnP G [j-,J+]) (E.3) 

and 

limsup-ilnpf4^e]j_,j+[') < ini Rx{j). (E.4) 

t \ t J je]j-,3+[ 
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The property that Rx is lower semicontinuous means that 

lim inf Rx{j') = Rx{j) for every j. (E.5) 

Notice that using an open interval ]j — e,j + e[ instead of [j — e,j + e] in this definition leads to 
the same notion. By taking j+ = j + e, j_ = j — e and letting e \ 0, we see that if Xt obeys a 
LDP then the rate function is given by 

Rx (j) = lim lim M -llnP ( ^ e \j - s,j + sU (E.6) 

e\0 t->-+oo t \ t ) 

= hm lim sup — - In P [ — e] j — e, j + £[ J . 

Note that we do not claim that the existence of the above limits, equal to some lower semi- 
continuous function Rx{j), guarantees that Xf satisfies a LDP. The above definitions involve 
lim sup and lim inf and make a difference between and to take care of slightly 

pathological situations. 

However, if Xt satisfies a LDP with a continuous rate function, then infjg]j_ = 
mfj^[j_j^]Rx{j), the lim sup and lim inf become standard limits and the inequalities become 
equalities : 

hm -UnP = lim -^InP G [j-,J+]) = . inf. RxU)- (E.7) 

One can rewrite this informally as 

p( — en ~ g-tinf.e/iJxO) (E.8) 

\ t ) t^+oa 

whenever 7 is an interval. More and more informal, and possibly misleading, statements are 

pf^G[j,J+rfj]) - e-*«-W or p[^ = ^ ^ e-*«-«. (E.9) 

\ t J t— >+oo y t J t-^+oo 

Before we turn to other possible characterizations of large deviations, let us explain briefly and 
heuristically the presence of the inf Rx{j) in these formulae. If the interval / is split into a finite 
number of intervals, say I = U^^/fc, we have 



max 

k 



P (f e 4) < P e /) < e /.) < i^maxP e 4) . (E.io) 



We infer that 
SO that, for s >t, 

- ^+min-ilnpf^ e/fc) <-ilnpf^e/) < min In P G 4 V (E.12) 
First we take the inf over s>t and interchange freely the inf and the min to get 
- +mininf--lnP ( — G 4 | < inf-^lnP ( — G J | < mininf-ilnP | — G 4 | . 

t k s>t S \ S J s>t S \ S J k s>t S \ S J 

(E.13) 
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Letting t +00 yields 



liminf-ilnP ( — e / ) = min lim inf - ^ In P f — € 4 ) . (E.14) 

Second we take the sup over s>t. This time only one of the inequalities survives the interchange 
of sup and min 

sup--lnP ( — e / ) < supmin--lnP ( — e h] < minsup--lnP ( — e 4 ) (E.15) 

s>t S \ S J s>t k S \ S J k s>t S \ S ) 

and letting t +00 yields 

lim sup —i In P ( — el) < min lim sup —- InP [ — e 4 ) . (E.16) 

To summarize 

minliminf-ilnP I — G 4 ) = liminf -i InP f — € / ) (E.17) 

1 /X \ 1 /X 
lim sup In P f — € / I < min lim sup In P f — £ 4 

t-s-+oo t \ t J k t_).+oo t \ t 

Informally, this implies indeed that the large deviation estimates for the interval / come solely 
from some arbitrary small subinterval of / characterized by a minimizing property. 



E.2 Alternative definitions 

There are a number of situations of physical interest when Xt takes values in a (time independent) 
discrete set S. Typically S is Z, the set of integers, or a + Z, the set of integers shifted by a, or 
aZ, the set of integers dilated by a. 

Though this discreteness by no means prevents from using the general large deviation theory, 
other reasonable but ad hoc definitions come to mind. 

For instance, one may ask whether there is a function gx{j) from IR to [0, +00] such that, for 
any map t ^ St from [0, +oo[ to S such that limt_,.+(x) ^ = i, the limit 

lim -\\aP{Xt = St) (E.18) 

exists and has value gx{j)- 

A weaker condition comes from asking whether there is a function hx{j) from R to [0,+oo] 
such that the limit 

lim --\nP(Xt=tj) (E.19) 

t— >+oo t 

tjes 

exists and has value hx{j)- 

Of course same care is needed when choosing S, which would better be in some way minimal. 
For instance, if you take S = Z while Xt takes only even values, hx or gx have no chance to exist. 
However, when S is chosen appropriately and P{Xt = s) behaves nicely for large t and s, it can 
be expected on physical grounds that all three functions Rx, gx and hx exist and arc the same. 
Alas, we are not aware of practical and general enough mathematical criteria that guarantee this 
fact. Of the three, hx is probably the easiest to tackle with bare hands. However, to deal with Rx 
one can often rely on powerful theorems of the general theory of large deviations. But whether 
they exist (and coincide) or not, the three functions are meaningful characteristics of the large 
deviations of Xf. With some imagination, we could probably invent others. 
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E.3 Inequalities at finite time and fiuctuation relation between large 
deviation functions 

One important feature of large deviation functions in out-of-equilibrium statistical mechanics is 
that they satisfy, under appropriate circumstances, symmetry relations. Usually, these relations 
are consequences of a, possibly generalized, time reversal symmetry. Typically, for some < m < 
M < +00 and for some real 7 one has 

mP{Xt = -s)e'^^ < P{Xt = s) < MP{Xt = -s)e^" (E.20) 

for every s. To be fair, this way of writing things applies, strictly speaking, only when Xt is 
atomic, i.e. takes values in a discrete set. A more correct statement is that the laws of Xt and 
—Xt are absolutely continuous and the Radon-Nikodym derivativ^ 

me^^ < f^^'^f^, < Me'^^ (E.21) 
dP-Xt (s) 

of which the previous equation is a special case. 

The point we want to make here is that, with any of the three definitions of large deviation 
functions given above (when they do exist), the above equation implies a symmetry relation. For 
gx or hx we can use the simplified equation evaluated when s € S. It implies 

--lnM-j---lnP{Xt = -s) < --\nP{Xt = s) < --\nm--f---\nP{Xt = -s). (E.22) 

Assuming the existence of hx one can take the limit limt->.+oo to get hx{-~i) — 7J < hxij) < 

tjes 

hx{-j) - 7i: i-e- 

hx{])^hx{-j)-lj, (E.23) 

the announced symmetry relation. If gx exists, then so does hx and they are equal, so the 
symmetry property for gx (when it exists) is a consequence of the above bounds. In the case 
when Rx exists, the proof of its symmetry based on the inequalities for the Radon-Nikodym 
derivative is only slightly more elaborate : if B is a Borel subset of IR with <j_ < inf B and 
sup B < tj+ then, assuming 7 > for definiteness, 

P{-Xt e B)e<^^- < [ dP-xAsV" < P{-Xt e B)e^^^+ (E.24) 
Jb 



so, as a consequence of (jE.21|) . 

ilnM-7j+ - - \nP{Xt G -B) < --lnP{Xt e B) < --\nm-jj^ - - \nP{Xt G -B) 

(E.25) 



and taking B ^-^d the limsupt_>.^3Q in the second inequality we get 

lim sup - y In P ( ^ e] j_ , j+ [ ) < -7j_ + lim sup - y In P ( ^ e] - j+ , - j_ [ ) . (E.26) 

Similarly, taking B — the liminft_i._|_oo of the first inequality we get 

-7J+ +liminf-ilnP S [-J+,-J-]) < liminf-ylnP e ] . (E.27) 

t->-|-oo t \ t I i-!- + oo t \ t 



Taking j± — j ± e, j fixed, e \ we find from the characterization (jE.6P of Rx 

Rx (j) < -IJ + Rx i-j) and - 7j + Rx {-j) < Rx (j) (E.28) 



^One says that the Radon-Nikodym derivative of a probability measure fi writh respect to a probabiUty measure 
1/ exists if there is an ;^-integrable function / such that for every Borel set B one has fg dfi = fg fdv. Then / is 
called the Radon-Nikodym derivative of with respect to v and one writes / = dfi/du. 



52 



so 

RxU)^Rx{-j)~lJ, (E.29) 

the announced symmetry. So with respect to fluctuation relations, our three definitions of large 
deviations are on the same footing. 

We notice that the above relation is usually written in the more symmetric form Rx{j) ~ 
Rx{—j) = — 7J, which is however ambiguous when Rx{j) is infinite. 



E.4 Cumulated quantities with sub-extensive difference 

We would like to stress that some natural and desirable properties of large deviation functions are 
not automatic with some of our definitions. 

For instance, suppose that Xt and Yt are random processes with a sub-extensive difference, i.e. 
there is a non-random function Ct such that lim(_>+oo Ct = and jXj — Ft| < tCt- Then it should 
be expected that the following alternative holds : 

{- Xf and Yf have the same large deviation function ,^ 
. . . (E.30) 

- neither Xt nor Yt has a large deviation function. 

We shall show that this holds true within the general theory of large deviations. With our ad-hoc 
definitions, we can see that even when Xt and Yt have the required properties, the discrete sets on 
which Xt and Yt take their values can be very different. And even if the relation is simple, some 
problems remain at least with the weakest definition of large deviation functions. 

To make a proof within the general theory of large deviations, we note that, if j_ < j+, any 
real function R has the property that 

hm inf R{j)^ inf R{j). (E.31) 

Moreover, if R is lower semi-continuous then 

hm inf R{j)= inf R{j). (E.32) 

Now, let e > 0. For s large enough, < £ by hypothesis, so that 

whenever t is such that Cs < e ioT s > t. Then 

M--lnp(^ e[j^-e,j+ + e]) < mi -- \n P e [j-, , (E.34) 

s>t S \ S J s>t S \ S J 

SO taking t. +00 one gets 

liminf-ilnP ( — e [j_ +£] ) < liminf -i InP ( — e j+] ) . (E.35) 
t \ t J t \ t J 

Assuming Xt has a large deviation function, the left-hand side is > mij^y__^ j^_^_^■^ Rx{j) by 
(|E.3|) . so we have proved that for any e > one has 

inf Rx U) < hm inf - i In P f ^ e [j_ , . (E.36) 

In the same vein, assuming j_ < j+, one has, for e small enough (e < ''''' ) 

P e]3- + e,j+ - e] \ <P(^ for ,s > t (E.37) 
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whenever t is such that Cs < e for s > t. As above, we infer that 

hmsup-ihiP < hmsup-ihiP e]]- + e,j+ - e[) . (E.38) 

Assuming Xt has a large deviation function, the right-hand side is < infjg]j_+j Px(j) by 
(jE.4p . so for e small enough one has 

hmsup-ilnpf^ eb^.j+i) < inf PxO') (E.39) 

Taking the limit e \ in (|E.36p and (|E.39p (using the lower semi-continuity of Rx in (|E.36|) ) 
we obtain precisely that Yt satisfies an LDP (jE.3p - (jE.4p with large deviation function Rx- This 
proves that the alternative (jE.30p holds. 

We conclude this discussion with a (counter)-example which has the advantage of being ex- 
tremely simple, but the drawback that it is artificial (in particular, it is not continuous in proba- 
bility, it even has non-random discontinuity times). The example is Xt = \t\ the integer part of 
t, i.e. \t\ is an integer and [ij < t < [tj + 1. 

We urge the reader to check that 

- Xt has a large deviation function Rx and a large deviation function hx which coincide, 
namely 

Rx{])^hx{j)^r^ 'l\Z\- (^-40) 

[+00 if J 7^1 

- liYt = Xt '\- \ then Yt has a large deviation function Ry {— Rx by the previous result) but 
hy, while existing, differs from hx^ namely 

hyii) = +00 whatever j. (E.41) 

- Xt (resp. Yt) has no large deviation function gx (resp. gy)- 

Here are proofs for the two first properties. The third is easy and left to the reader. 
The case of hx '■ we take S = IN 

lim --lnP{Xt = tj) = lim -- Inlu (E.42) 

If j > 1 [ij < i < tj so tyt\=tj = 0. li j < I \t\ > t ~ I > tj whenever t > y^j, so tyt\=tj = for 
t large enough. If j = 1 IL|^jj^j = 1 for t e N. This gives the announced formula (jE.40p for hx- 

The case oi Rx ■ P = Hs\e[s]-,s3+]- 

-Ifl e and s e N then P (^ e [j-,J+]) = 1, so for any t inf,>t -i In P (^^ G =0 

and liminft^+oo - i InP G []-,]+]) = 0. 

- If j- > 1 then [sj < sj- whenever s > so for any i > inf^x — InP e = +oo 
and liminff^+oo -j InP (^ G [j-, J+]) = 



-CXD. 



If j+ < 1 then [sJ > s— 1 > sj+ whenever s > , so inf^x — InP (^ G j+]) = +oo 



whenever t > jzj^ ^-i^d liminfj^+oo — 7 InP G [j-,i+]) = +00. 
To summarize 

t^+co t \ t J \ +00 else 

Analogously P e]j-,j+[) - 1lsJg]sj_,sj+[- As P < ^ e b-, we have 

hminf-ilnP G [j-,J+]] < limsup-^lnP e]j-,j+[) . (E.44) 

t V * / t-*-|-oo t \ t J 
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As the left-hand side is +00 if 1 ^ so the right-hand side must be 4-00 if 1 ^ 

and in particular if 1 On the other hand, if 1 then [sj g]sj-,sj+[ as soon as 

s > -^J^. so limsupj_j.^g^ — J InP = 0- To summarize 

limsup-ilnpf^eb-_,,,[) ^{^ ifleb--,jV[, 
i^+oc I \ I / 1+00 else 

From (|E.45|) . (|E.43I) and the definition of an LDP (IE.3p - p.4l) we infer that Xt has a large deviation 
function 

RAj)-\1 (E.46) 
I +00 else 

and Rx = hx- The fact that Xt has no large deviation function gx is straightforward. 
The case of hy ■ we take S = N* 

1. „ 1 



-lnP(ri = ij)= lim --lnlLtj+i=t,. (E.47) 



lim 



If j > 1 [ij < t + 1 < tj whenever t > -pj and then iLfj+i^fj =0. If j < 1 then [t\+l > t > tj for 
every t and then — 0. So limt-i.+oo — j In P(Yi = tj) = +00 for every j and hyij) — -f-00 

for every j as announced in (|E.4ip . 
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